LVS-DR mode + Keepalived theory, super detailed!

One: LVS-DR working principle

1.1: Overview of DR mode

• Load balancing cluster working mode-direct routing (Direct Routing)
  • Referred to as DR mode, it adopts a semi-open network structure, which is similar to the structure of TUN mode, but the nodes are not scattered everywhere, but are located on the same physical network as the scheduler
  • The load scheduler is connected to each node server through the local network, without the need to establish a dedicated IP tunnel

1.2: Data packet flow analysis

• In order to facilitate the principle analysis, put the Client and the cluster machine on the same network, and the route of the data packet flow is 1-2-3-4

1. The Client sends a request to the target VIP, and the Director (load balancer) receives it.

• IP header and data frame header information

2. Director selects Realserver_1 according to the load balancing algorithm, does not modify or encapsulate the IP message, but changes the MAC address of the data frame to the MAC address of RealServer_1, and then sends it on the LAN.

• IP header and data frame header information

3. RealServer_1 receives this frame and finds that the target IP matches the local machine after decapsulation (RealServer is bound to VIP in advance), so it processes this message. Then re-encapsulate the message and send it to the LAN.

4. The Client will receive the reply message. Client thinks that it is getting normal service, but does not know which server handles it

Note: If it crosses the network segment, the message will be returned to the user via the router via terne.

5. Summary: For the same IP address, the MAC address is changed

1.3: ARP problems in LVS-DR

• In the LVS-DR load balancing cluster, the load balancer and node server must be configured with the same VIP address

• Having the same IP address in the local area network will inevitably cause disorder in the ARP communication of each server
  • When an ARP broadcast is sent to the LVS-DR cluster, because the load balancer and the node server are connected to the same network, they will both receive the ARP broadcast
  • At this time, only the front-end load balancer responds, and other node servers should not respond to ARP broadcasts
• Process the node server so that it does not respond to ARP requests for VIP
  • Use virtual interface lo:0 to carry VIP address
  • Set the kernel parameter arp_ignore=1: the system only responds to ARP requests whose destination IP is the local IP
• RealServe return packets (the source IP is VIP) are forwarded by the router, and the MAC address of the router must be obtained first when re-encapsulating the packet

• When sending an ARP request, Linux defaults to use the source IP address of the IP packet (ie VIP) as the source IP address in the ARP request packet instead of the IP address of the sending interface (such as ens33)

• After the router receives the ARP request, it will update the ARP table entry
• The original VIP corresponding to the Director's MAC address will be updated to the ⅥP corresponding to the MAC address of the RealServer

• At this time, the router will forward the new request message to RealServer according to the ARP table entry, resulting in

Director's VIP is invalid

• Solution

Process the node server and set the kernel parameter arp_announce=2: The system does not use the source address of the IP packet to set the source address of the ARP request, but selects the IP address of the sending interface

1.4: Methods to solve the above two ARP problems

- '修改/etc/sysctl.conf文件'
  - '对节点服务器进行处理，使其不响应针对VIP的ARP请求'
  - net.ipv4.conf.lo.arp_ignore = 1
  - net.ipv4.conf.lo.arp_announce = 2
  - '系统不使用IP包的源地址来设置ARP请求的源地址，而选择发送接口的IP地址'
  - net.ipv4.conf.all.arp_ignore = 1
  - net.ipv4.conf.all.arp_announce = 2

Two: keepalived realization principle

2.1: Keepalived case analysis

• In enterprise applications, a single server bears the risk of a single point of failure in the application
• Once a single point of failure occurs, enterprise services will be interrupted, causing great harm

2.2: Overview of keepalived tools

• A health check tool specially designed for LVS and HA
  • Support automatic failover (Failover)
  • Support node health check (Health Checking)
  • Official website http://www.keepalived.org

2.3: Principle analysis

• Keepalived adopts VRRP hot backup protocol to realize the multi-machine hot backup function of Linux server

• VRRP, Virtual Routing Redundancy Protocol, is a backup solution for routers

• Multiple routers form a hot backup group, and provide services to the outside through a shared virtual IP address

• Only one main router provides service at the same time in each hot backup group, and other routers are in redundant state

• If the currently online router fails, other routers will automatically take over the virtual IP address according to the set priority and continue to provide services

Three: How to deploy Keepalievd

3.1: Overview of Keepalievd deployment

• Keepalived can realize multi-machine hot backup, each hot-standby group can have multiple servers, the most commonly used is dual-machine hot backup

• The failover of dual-system hot backup is realized by the drift of virtual IP address, which is suitable for various application servers

• This deployment will implement dual-machine hot backup based on web services

3.2: Keepalievd installation and startup

• When applied in the LVS cluster environment, the lipvsadm management tool is also needed
• YUM install Keepalived
• Enable Keepalived service

3.3: Configure Keepalievd master server

• Keepalievd configuration directory is located at /etc/keepalievd/
• keepalievd.conf is the main configuration file
  • global_defs{...} section specifies global parameters
  • The vrrp_instance instance name{...} section specifies the VRRP hot standby parameters
  • The comment text begins with the "!" symbol
  • Directory samples/, provides many configuration samples as reference
• Common configuration options
  • router_id HA_TEST_R1: The name of the router (server)
  • vrrp_instance VI_1: Define VRRP hot standby instance
  • state MASTER: Hot standby state, MASTER represents the master server
  • interface ens33: the physical interface that carries the VIP address
  • virtual_router_id 1: The ID number of the virtual router, which is consistent for each hot standby group
  • priority 100: priority, the larger the value, the higher the priority
  • advert_int 1: The number of seconds between notifications (heartbeat frequency)
  • auth_type PASS: authentication type
  • auth_pass 123456: password string
  • virtual_ipaddress{vip}: Specify a drift address (VIP), there can be multiple, multiple drift addresses are separated by commas

3.4: Configure Keepalived slave server

• There are three options for the configuration of the Keepalived backup server and the master configuration.
  • router_id: set as free name
  • state: set to BACKUP
  • priority: value is lower than the primary server
• Other options are the same as master

3.5: Keepalived dual-system hot backup effect test

• Test the effect of dual machine hot backup
  • Web services are enabled on the main and standby machines, and the content is the same
  • Disable and enable the network card of the main server successively, and perform the following tests
• Test 1: Use ping to detect the connectivity of 19216810.72
• Test 2: Visit htt:/192168.10.72, confirm the usability and content changes of the
  card, and perform the following tests
• Test 1: Use ping to detect the connectivity of 19216810.72
• Test 2: Visit htt:/192168.10.72 to confirm availability and content changes
• Test 3: View the changes in the log file /var/log/messages