LVS-NAT：搭建HTTP及HTTPS负载均衡集群
1.环境说明：
| 所需设备 | IP地址 | VIP |
|---|---|---|
| LVS服务器（DR） | DIP：192.168.159.135 | VIP：192.168.159.250 |
| apache服务器（RS1） | IP：192.168.159.132 | VIP：192.168.159.250 |
| apache服务器（RS2） | IP：192.168.159.136 | VIP：192.168.159.250 |
| 客户端（longnian） | IP：192.168.159.137 | 无 |
服务器的系统均为centos7,三台服务器分别配置了对应的本地静态地址DIP和RIP，且在一个内网中。
LVS有两块网卡，一块IP是内网的DIP，另一块IP是公网IP的VIP
客户端的IP和VIP都是公网IP
RS的网关指向DIP
2.搭建NAT模式的HTTP负载集群
2.1在客户端上配置CIP
[root@longnian ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:b2:1a:0e brd ff:ff:ff:ff:ff:ff
inet 192.168.159.137/24 brd 192.168.159.255 scope global dynamic eno16777736
valid_lft 1567sec preferred_lft 1567sec
inet6 fe80::20c:29ff:feb2:1a0e/64 scope link
valid_lft forever preferred_lft forever
2.2在DR上配置DIP和VIP
[root@DR ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet 192.168.159.250/32 scope global lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:be:2f:65 brd ff:ff:ff:ff:ff:ff
inet 192.168.159.135/24 brd 192.168.159.255 scope global dynamic ens33
valid_lft 1103sec preferred_lft 1103sec
inet6 fe80::20c:29ff:febe:2f65/64 scope link
valid_lft forever preferred_lft forever
2.3在DR上开启IP转发，配置转发规则
//开启IP转发
[root@DR ~]# echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.conf
[root@DR ~]# sysctl -p
net.ipv4.ip_forward = 1
//配置规则
[root@DR ~]# yum -y install ipvsadm
[root@DR ~]# ipvsadm -A -t 192.168.159.250:80 -s rr
[root@DR ~]# ipvsadm -a -t 192.168.159.250:80 -r 192.168.159.132:80 -m
[root@DR ~]# ipvsadm -a -t 192.168.159.250:80 -r 192.168.159.136:80 -m
[root@DR ~]# ipvsadm -Sn > /etc/sysconfig/ipvsadm
//查看规则
[root@DR ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.159.250:80 wrr
-> 192.168.159.132:80 Route 1 0 0
-> 192.168.159.136:80 Route 1 0 0
2.4在RS上配置RIP，并将网关指向DIP
[root@RS1 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet 192.168.159.250/32 scope global lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:4a:7e:86 brd ff:ff:ff:ff:ff:ff
inet 192.168.159.132/24 brd 192.168.159.255 scope global dynamic eno16777736
valid_lft 1503sec preferred_lft 1503sec
inet6 fe80::20c:29ff:fe4a:7e86/64 scope link
valid_lft forever preferred_lft forever
[root@RS2 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet 192.168.159.250/32 scope global lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:2e:bc:ec brd ff:ff:ff:ff:ff:ff
inet 192.168.159.136/24 brd 192.168.159.255 scope global dynamic eno16777736
valid_lft 1418sec preferred_lft 1418sec
inet6 fe80::20c:29ff:fe2e:bcec/64 scope link
valid_lft forever preferred_lft forever
2.5在RS上配置HTTP
[root@RS1 ~]# yum -y install httpd
[root@RS1 ~]# cd /var/www/html/
[root@RS1 html]# echo 'This is RS1' > index.html
[root@RS1 html]# systemctl start httpd
[root@RS2 ~]# yum -y install httpd
[root@RS2 ~]# cd /var/www/html/
[root@RS2 html]# echo 'This is RS2' > index.html
[root@RS2 html]# systemctl start httpd
2.6客户端访问验证
[root@longnian ~]# for i in $(seq 10);do curl 192.168.159.250;done
This is RS1
This is RS2
This is RS1
This is RS2
This is RS1
This is RS2
This is RS1
This is RS2
This is RS1
This is RS2
3.搭建HTTPS负载均衡集群
3.1生成证书
//生成一对密钥
[root@DR ~]# cd /etc/pki/CA/
[root@DR CA]# (umask 077;openssl genrsa -out private/cakey.pem 2048)
Generating RSA private key, 2048 bit long modulus
.......+++
.......+++
e is 65537 (0x10001)
//生成自签署证书
[root@DR CA]# openssl req -new -x509 -key private/cakey.pem -out cacert.pem -days 365
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:CN
State or Province Name (full name) []:HB
Locality Name (eg, city) [Default City]:WH
Organization Name (eg, company) [Default Company Ltd]:www.a.com
Organizational Unit Name (eg, section) []:www.a.com
Common Name (eg, your name or your server's hostname) []:www.a.com
Email Address []:[email protected]
//在RS生成证书签署请求，并发送给CA
[root@RS1 ~]# mkdir /etc/httpd/ssl
[root@RS1 ~]# cd /etc/httpd/ssl
[root@RS1 ssl]# (umask 077;openssl genrsa -out httpd.key 2048)
Generating RSA private key, 2048 bit long modulus
...........................................................................................................+++
................................................+++
e is 65537 (0x10001)
[root@RS1 ssl]# openssl req -new -key httpd.key -days 1024 -out httpd.csr
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:CN
State or Province Name (full name) []:HB
Locality Name (eg, city) [Default City]:WH
Organization Name (eg, company) [Default Company Ltd]:www.a.com
Organizational Unit Name (eg, section) []:www.a.com
Common Name (eg, your name or your server's hostname) []:www.a.com
Email Address []:[email protected]
Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:
[root@RS1 ssl]# scp httpd.csr [email protected]:/root/
The authenticity of host '192.168.159.135 (192.168.159.135)' can't be established.
ECDSA key fingerprint is 03:18:89:32:f3:e7:3e:8b:44:2a:9c:ed:0a:8f:53:6f.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.159.135' (ECDSA) to the list of known hosts.
[email protected]'s password:
httpd.csr 100% 1033 1.0KB/s 00:00
//CA签署证书并发给客户端
[root@DR ~]# openssl ca -in /root/httpd.csr -out httpd.crt -days 365
Using configuration from /etc/pki/tls/openssl.cnf
Check that the request matches the signature
Signature ok
Certificate Details:
Serial Number: 2 (0x2)
Validity
Not Before: Jul 24 11:25:38 2020 GMT
Not After : Jul 24 11:25:38 2021 GMT
Subject:
countryName = CN
stateOrProvinceName = HB
organizationName = www.a.com
organizationalUnitName = www.a.com
commonName = www.a.com
emailAddress = [email protected]
X509v3 extensions:
X509v3 Basic Constraints:
CA:FALSE
Netscape Comment:
OpenSSL Generated Certificate
X509v3 Subject Key Identifier:
AC:14:97:6A:0F:9C:62:70:9A:C9:4C:AD:C6:0E:EC:63:93:67:A4:44
X509v3 Authority Key Identifier:
keyid:16:FC:92:C6:94:8B:B8:31:B4:CE:12:EF:43:E4:7D:D1:C5:95:00:C6
Certificate is to be certified until Jul 24 11:25:38 2021 GMT (365 days)
Sign the certificate? [y/n]:y
1 out of 1 certificate requests certified, commit? [y/n]y
Write out database with 1 new entries
Data Base Updated
[root@DR ~]# ls
httpd.crt httpd.csr
//CA把签署好的证书httpd.crt和服务端的证书cacert.pem发给客户端
[root@DR ~]# scp httpd.crt [email protected]:/etc/httpd/ssl
[email protected]'s password:
httpd.crt 100% 4565 4.5KB/s 00:00
[root@DR ~]# scp httpd.crt [email protected]:/etc/httpd/ssl
The authenticity of host '192.168.159.136 (192.168.159.136)' can't be established.
ECDSA key fingerprint is 3d:0e:f3:00:71:87:a5:a0:ec:a4:64:95:e0:bc:78:2b.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.159.136' (ECDSA) to the list of known hosts.
[email protected]'s password:
httpd.crt 100% 4565 4.5KB/s 00:00
[root@DR ~]# scp /etc/pki/CA/cacert.pem [email protected]:/etc/httpd/ssl
[email protected]'s password:
cacert.pem 100% 1383 1.4KB/s 00:00
3.2配置HTTPS
在RS1上将httpd.key传给RS2,并在RS上安装ssl模块
[root@RS1 ~]# cd /etc/httpd/ssl/
[root@RS1 ssl]# ls
httpd.crt httpd.csr httpd.key
[root@RS1 ssl]# scp httpd.key [email protected]:/etc/httpd/ssl/
The authenticity of host '192.168.159.136 (192.168.159.136)' can't be established.
ECDSA key fingerprint is 3d:0e:f3:00:71:87:a5:a0:ec:a4:64:95:e0:bc:78:2b.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.159.136' (ECDSA) to the list of known hosts.
[email protected]'s password:
httpd.key 100% 1679 1.6KB/s 00:00
[root@RS1 ssl]# yum -y install mod_ssl
[root@RS2 ssl]# ls
cacert.pem httpd.crt httpd.key
[root@RS2 ssl]# yum -y install mod_ssl
在RS上编辑配置文件
[root@RS1 ~]# vim /etc/httpd/conf.d/ssl.conf
<VirtualHost _default_:443>
# General setup for the virtual host, inherited from global configuration
//将下面两行删除注释，并修改域名
DocumentRoot "/var/www/html"
ServerName aaa.com:443
...
//修改下面没有带注释的行
# certificate can be generated using the genkey(1) command.
SSLCertificateFile /etc/httpd/ssl/httpd.crt
# Server Private Key:
# If the key is not combined with the certificate, use this
# directive to point at the key file. Keep in mind that if
# you've both a RSA and a DSA private key you can configure
# both in parallel (to also allow the use of DSA ciphers, etc.)
SSLCertificateKeyFile /etc/httpd/ssl/httpd.key
...
...
# Certificate Authority (CA):
# Set the CA certificate verification path where to find CA
# certificates for client authentication or alternatively one
# huge file containing all of them (file must be PEM encoded)
SSLCACertificateFile /etc/httpd/ssl/cacert.pem
[root@RS1 ~]# systemctl restart httpd
//RS2同上.....
3.3在DR上配置规则
[root@DR ~]# ipvsadm -A -t 192.168.159.250:443 -s rr
[root@DR ~]# ipvsadm -a -t 192.168.159.250:443 -r 192.168.159.132 -m
[root@DR ~]# ipvsadm -a -t 192.168.159.250:443 -r 192.168.159.136 -m
[root@DR ~]# ipvsadm -Sn > /etc/sysconfig/ipvsadm
[root@DR ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.159.250:80 wrr
-> 192.168.159.132:80 Route 1 0 0
-> 192.168.159.136:80 Route 1 0 0
TCP 192.168.159.250:443 rr
-> 192.168.159.132:443 Masq 1 0 0
-> 192.168.159.136:443 Masq 1 0 0
TCP 192.168.159.250:3306 rr
-> 192.168.159.132:3306 Route 1 0 0
-> 192.168.159.136:3306 Route 1 0 0
4.验证
[root@longnian ~]# for i in $(seq 10);do curl -k https://192.168.159.250;done
This is RS1
This is RS2
This is RS1
This is RS2
This is RS1
This is RS2
This is RS1
This is RS2
This is RS1
This is RS2