Tunnel mode
client -> vs-> Rs-> cllient
principles and characteristics:
Rationale:
1, when a user request arrives Director Server, this time the requested data packet will be first-strand PREROUTING kernel space. At this point the packet source IP CIP, the target IP as VIP.
2, PREROUTING examination revealed certain IP packets are native, the data packet to the INPUT chain
3, IPVS than packets requested service is a cluster service, if, in the request packet header encapsulating an IP packet again text, as the source IP encapsulated DIP, to the destination IP RIP. Then sent to POSTROUTING chain. At this time, the source IP DIP, destination IP of the RIP
. 4, the POSTROUTING chain according to the latest encapsulated IP packet, the data packet sent to RS (since the outer layer of a multi-package IP header, this time can be understood as to tunnel ). At this time, the source IP DIP, to the destination IP RIP
after 5, RS received message is found its own IP address, the packet will be received down, removal off the outermost IP, you will find that there is also a layer of IP header and its goal is the VIP lo interfaces, then the time RS start processing the request, after completion of the processing, sent via the interface eth0 lo card, and then transfer out. At this time, the source IP address of the VIP, the destination IP is the CIP
. 6, the final response packet delivered to a client
LVS-Tun model characteristics
- RIP, VIP, DIP all public addresses
- RS gateway not and can not point to DIP
- All request messages via Director Server, but the response packet must not been to the Director Server
does not support port mapping - RS system must support tunneling
LVS (tunnel mode tun) mode to build
Note: In actual production RIP, VIP, DIP all public addresses, because the current restrictions, in order to achieve the experimental results. Experiments RIP, VIP, DIP, CIP all of the same network segment ip.
When the experiments carried out on the following host 7.3
host kernel must support tunnel mode can only be used in this manner.
TUNM LVS configuration mode:
server1 server2, server3: add a tunnel, tunnel adding the external exposure of VIP, tunnel activation
[root@server1 ~]# modprobe ipip #加载模块
[root@server1 ~]# ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:d9:67:4d brd ff:ff:ff:ff:ff:ff
inet 172.25.46.1/24 brd 172.25.46.255 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::5054:ff:fed9:674d/64 scope link
valid_lft forever preferred_lft forever
3: tunl0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN qlen 1 # 加载成功出现tunl0接口
link/ipip 0.0.0.0 brd 0.0.0.0
[root@server1 ~]# ip addr add 172.25.46.150/24 dev tunl0 #添加vip
[root@server1 ~]# ip link set up tunl0 # 激活网络接口
[root@server1 ~]# ip addr show
3: tunl0@NONE: <NOARP,UP,LOWER_UP> mtu 1480 qdisc noqueue state UNKNOWN qlen 1
link/ipip 0.0.0.0 brd 0.0.0.0 #此处不是down表示已经激活
inet 172.25.46.150/24 scope global tunl0
valid_lft forever preferred_lft forever
server2:
[root@server2 ~]# modprobe ipip
[root@server2 ~]# ip addr show
[root@server2 ~]# ip addr add 172.25.46.150/24 dev tunl0
[root@server2 ~]# ip link set up tunl0
[root@server2 ~]# ip addr show
server3:
[root@server3 ~]#modprobe ipip
[root@server3 ~]# ip addr show
[root@server3 ~]# ip addr add 172.25.46.150/24 dev tunl0
[root@server3 ~]# ip link set up tunl0
[root@server3 ~]# ip addr show
Add Policy:
[root@server1 ~]# yum install -y ipvsadm ##安装ipvsamd 如果之前已经安过可以用ipvsadm -C 清除之前做的策略
[root@server1 ~]# ipvsadm -A -t 172.25.46.150:80 -s rr #添加vip ,对后端服务器采用rr算法
[root@server1 ~]# ipvsadm -a -t 172.25.46.150:80 -r 172.25.46.2:80 -i ##添加后端真实的服务器
[root@server1 ~]# ipvsadm -a -t 172.25.46.150:80 -r 172.25.46.3:80 -i
[root@server1 ~]# ipvsadm -l
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP server1:http rr
-> 172.25.46.2:http Tunnel 1 0 0
-> 172.25.46.3:http Tunnel 1 0 0
server2, server3 install httpd service, and set up a shared page
to test the effect is obvious, different real server can set different share page content.
server2:
[root@server2 ~]#yum install -y httpd
[root@server2 ~]# cd /var/www/html/
[root@server2 html]# ls
index.html
[root@server2 html]# cat index.html
server2
[root@server2 html]# systemctl start httpd
server3:
[root@server3~]#yum install -y httpd
[root@server3 ~]# cd /var/www/html/
[root@server3 html]# ls
index.html
[root@server3 html]# cat index.html
server3
[root@server3 html]# systemctl start httpd
Close kernel reverse filtering (all real servers)
parameters grep rp_filter # filter kernel about filtering function | sysctl -a
server2:
[root@server2 html]# sysctl -a | grep rp_filter ##筛选内核有关过滤功能
[root@server2 html]# sysctl -w net.ipv4.conf.all.rp_filter=0 [root@server2 html]# sysctl -w net.ipv4.conf.default.rp_filter=0 ##将参数1 的都改为0
[root@server2 html]# sysctl -w net.ipv4.conf.eth0.rp_filter=0 [root@server2 html]# sysctl -w net.ipv4.conf.tunl0.rp_filter=0
[root@server2 html]# sysctl -a | grep rp_filter
server3:
[root@server3 html]# sysctl -a | grep rp_filter
net.ipv4.conf.all.arp_filter = 0
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.arp_filter = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.eth0.arp_filter = 0
net.ipv4.conf.eth0.rp_filter = 1
net.ipv4.conf.lo.arp_filter = 0
net.ipv4.conf.lo.rp_filter = 0
net.ipv4.conf.tunl0.arp_filter = 0
net.ipv4.conf.tunl0.rp_filter = 1
[root@server3 html]# sysctl -w net.ipv4.conf.all.rp_filter=0
net.ipv4.conf.all.rp_filter = 0
[root@server3 html]# sysctl -w net.ipv4.conf.default.rp_filter=0
net.ipv4.conf.default.rp_filter = 0
[root@server3 html]# sysctl -w net.ipv4.conf.eth0.rp_filter=0
net.ipv4.conf.eth0.rp_filter = 0
[root@server3 html]# sysctl -w net.ipv4.conf.tunl0.rp_filter=0
net.ipv4.conf.tunl0.rp_filter = 0
[root@server3 html]# sysctl -a | grep rp_filter
net.ipv4.conf.all.arp_filter = 0
net.ipv4.conf.all.rp_filter = 0
net.ipv4.conf.default.arp_filter = 0
net.ipv4.conf.default.rp_filter = 0
net.ipv4.conf.eth0.arp_filter = 0
net.ipv4.conf.eth0.rp_filter = 0
net.ipv4.conf.lo.arp_filter = 0
net.ipv4.conf.lo.rp_filter = 0
net.ipv4.conf.tunl0.arp_filter = 0
net.ipv4.conf.tunl0.rp_filter = 0
The purpose of modifying the parameters rp_filter
There are three values, 0, 1, specific meaning:
0: Not open source address check.
1: Enable reverse path strict check. For each incoming packet, check the reverse path is the best path. If the reverse path is not the best route, the packet is discarded directly.
2: Open loose reverse path verification. For each incoming packet, checking its source address is reachable, i.e., whether the reverse path can pass (by any network port), the reverse path if different, then discards the packet.
test:
[root@foundation46 ~]# curl 172.25.46.150
server2
[root@foundation46 ~]# curl 172.25.46.150
server3
[root@foundation46 ~]# curl 172.25.46.150
server2
[root@foundation46 ~]# curl 172.25.46.150
server3
[root@foundation46 ~]# curl 172.25.46.150
server2
[root@foundation46 ~]# curl 172.25.46.150
server3
[root@server1 ~]# ipvsadm
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP server1:http rr
-> 172.25.46.2:http Tunnel 1 0 3
-> 172.25.46.3:http Tunnel 1 0 3