Basic knowledge of LVS/DR working mode

Insert picture description here

Experimental instructions:
1. The network uses NAT mode.
2. The DR mode requires that Director DIP and all RealServer RIPs must be in the same network segment and broadcast domain
. 3. All node gateways are designated as real gateways

CPU name	ip	system	use
client	172.16.147.1	mac	Client
lvs-server	172.16.147.154	centos7.5	Distributor
real-server1	172.16.147.155	centos7.5	web1
real-server2	172.16.147.156	centos7.5	web2
vip for dr	172.16/147.200	(The real scene is the public network ip)

2. LVS/DR mode implementation

1. Preparations (all hosts in the cluster) close the firewall and selinux

[root@lvs-server ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
172.16.147.154 lvs-server
172.16.147.155 real-server1
172.16.147.156 real-server2

2. Director distributor configuration

Configure VIP

[root@lvs-server ~]# ip addr add dev ens33 172.16.147.200/32 #设置VIP
[root@lvs-server ~]# yum install -y ipvsadm   #RHEL确保LoadBalancer仓库可用
[root@lvs-server ~]# service ipvsadm start  #启动
注意:启动如果报错: /bin/bash: /etc/sysconfig/ipvsadm: 没有那个文件或目录
需要手动生成文件
[root@lvs-server ~]# ipvsadm --save > /etc/sysconfig/ipvsadm

Define LVS distribution strategy

-A：添加VIP
-t：用的是tcp协议
-a：添加的是lo的vip地址
-r：转发到realserverip
-s：算法
-L|-l –list #显示内核虚拟服务器表
--numeric, -n：#以数字形式输出地址和端口号
-g --gatewaying #指定LVS工作模式为直接路由器模式（也是LVS默认的模式）
-S -save #保存虚拟服务器规则到标准输出，输出为-R 选项可读的格式
rr：轮循
如果添加ip错了，删除命令如下:
# ip addr del 172.16.147.200 dev ens33

[root@lvs-server ~]# ipvsadm -C  #清除内核虚拟服务器表中的所有记录。
[root@lvs-server ~]# ipvsadm -A -t 172.16.147.200:80 -s rr 
[root@lvs-server ~]# ipvsadm -a -t 172.16.147.200:80 -r 172.16.147.155 -g 
[root@lvs-server ~]# ipvsadm -a -t 172.16.147.200:80 -r 172.16.147.156 -g  
[root@lvs-server ~]# service ipvsadm save #保存方式一，使用下面的保存方式，版本7已经不支持了
[root@lvs-server ~]# ipvsadm -S > /etc/sysconfig/ipvsadm  #保存方式二，保存到一个文件中
[root@lvs-server ~]# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  172.16.147.100:80 rr
  -> 172.16.147.155:80            Route   1      0          0         
  -> 172.16.147.156:80            Route   1      0          0         
     
[root@lvs-server ~]# ipvsadm -L -n       
[root@lvs-server ~]# ipvsadm -L -n --stats    #显示统计信息
1. Conns    (connections scheduled)  已经转发过的连接数
2. InPkts   (incoming packets)       入包个数
3. OutPkts  (outgoing packets)       出包个数
4. InBytes  (incoming bytes)         入流量（字节）  
5. OutBytes (outgoing bytes)         出流量（字节）
[root@lvs-server ~]# ipvsadm -L -n --rate	#看速率
1. CPS      (current connection rate)   每秒连接数
2. InPPS    (current in packet rate)    每秒的入包个数
3. OutPPS   (current out packet rate)   每秒的出包个数
4. InBPS    (current in byte rate)      每秒入流量（字节）
5. OutBPS   (current out byte rate)      每秒出流量（字节）

3. All RS configuration

Configure the website server and test all RS #In order to test the effect, provide different pages (the following two real-servers operate)

[root@real-server1 ~]# yum install -y nginx
[root@real-server1 ~]# echo "real-server1" > /usr/share/nginx/html/index.html
两台机器都安装，按顺序添加不同的主机名以示区分
[root@real-server1 ~]# ip addr add dev lo 172.16.147.200/32   #在lo接口上绑定VIP
[root@real-server1 ~]# echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore  #忽略arp广播
[root@real-server1 ~]# echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce #匹配精确ip地址回包
[root@real-server1 ~]# systemctl start nginx 
[root@real-server1 ~]# systemctl enable  nginx 
=============================================================================
因为：realServer的vip有了，接着就是同一个网段中拥有两个vip, 客户端在网关发送arp广播需找vip时需要让realServer不接受响应.  
解决：
echo 1 >/proc/sys/net/ipv4/conf/eth0/arp_ignore 
arp_ignore 设置为1，意味着当别人的arp请求过来的时候，如果接收的设备没有这个ip，就不做出响应(这个ip在lo上，lo不是接收设备的进口)
echo 2 >/proc/sys/net/ipv4/conf/eth0/arp_announce   
使用最好的ip来回应，什么是最好的ip？同一个网段内子网掩码最长的

4. Test

[root@client ~]# elinks -dump http://172.16.147.200

LVS/DR working mode experimental operation

1. Environmental preparation:

192.168.138.134 lvs-server-vip
192.168.138.131 lvs-server-dip
192.168.138.133 http-server-1-rip
192.168.138.132 http-server-2-rip

2. Configuration of lvs-server-vip

添加一个子网掩码比已经存在的ip地址长的vip
ip a a dev ens37 192.168.138.134/32
LVS配置
ipvsadm -A -t lvs-server-vip:80 -s rr
ipvsadm -a -t lvs-server-vip:80 -r http-server-1-rip:80 -g
ipvsadm -a -t lvs-server-vip:80 -r http-server-2-rip:80 -g
ipvsadm-save > /etc/sysconfig/ipvsadm
ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.138.134:80 rr
  -> 192.168.138.132:80           Route   1      0          0         
  -> 192.168.138.133:80           Route   1      0          0

3. The configuration of http-server-1

yum -y install nginx
echo "real-server1" > /usr/share/nginx/html/index/html
在lo接口上绑定vip
ip a a dev lo 192.168.138.134/32
临时忽略arp广播
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
匹配精准ip地址回包
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
启动nginx
nginx

4. The configuration of http-server-2

yum -y install nginx
echo "real-server2" > /usr/share/nginx/html/index/html
在lo接口上绑定vip
ip a a dev lo 192.168.138.134/32
临时忽略arp广播
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
匹配精准ip地址回包
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
启动nginx
nginx