Nginx four-layer load balancing practice

1. What is four-layer load balancing

The four-layer load balancing is encapsulated based on the transport layer protocol packets (such as: TCP / IP), then the seven-layer we used earlier refers to the application layer, which is assembled on the basis of the four-layer, regardless of the four-layer or The seven layers refer to the OSI network model.

2. Four-layer load balancing application scenario

1、四层+七层来做负载均衡，四层可以保证七层的负载均衡的高可用性；如：nginx就无法保证自己的服务高可用，需 要依赖LVS或者keepalive。 
2、如：tcp协议的负载均衡，有些请求是TCP协议的（mysql、ssh），或者说这些请求只需要使用四层进行端口的转发 就可以了，所以使用四层负载均衡。

Four-layer + seven-layer construction scenarios for large-scale cluster architecture

3. Four-layer load balancing summary

• Layer 4 load balancing can only forward TCP / IP protocol, UDP protocol, usually used to forward ports, such as: tcp / 22, udp / 53;

• Layer 4 load balancing can be used to solve the port limitation problem of layer 7 load balancing; (layer 7 load balancing uses a maximum of 65535 port numbers)

• Four-layer load balancing can solve the high-availability problem of seven-layer load balancing; (multiple back-end seven-layer load balancing can be used by colleagues)

• The forwarding efficiency of layer 4 is much higher than that of layer 7, but it only supports tcp / ip protocol and does not support http and https protocols;

• Usually large concurrency scenarios usually choose to add a layer 4 load balance before the layer 7 load.

4. Practice of Nginx four-layer load balancing scenario

Nginx如何配置四层负载均衡

1、通过访问负载均衡的5555端口，实际是后端的web01的22端口在提供服务； 
2、通过访问负载均衡的6666端口，实际是后端的mysql的3306端口在提供服务。

先配置两台lb负载均衡

[root@lb02 ~]# cat /etc/yum.repos.d/nginx.repo   
[nginx‐stable]  name=nginx stable repo baseurl=http://nginx.org/packages/centos/7/$basearch/  
gpgcheck=0  
enabled=1 
gpgkey=https://nginx.org/keys/nginx_signing.key 
    
#在lb02上安装nginx 
[root@lb02 yum.repos.d]# yum install ‐y nginx 
    
#在lb02上同步lb01的所有nginx相关配置  
[root@lb02 ~]# scp ‐r [email protected]:/etc/nginx /etc/
    
#启动nginx 
[root@lb02 conf.d]# nginx ‐t  
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok 
nginx: configuration file /etc/nginx/nginx.conf test is successful  
    
[root@lb02 conf.d]# systemctl enable nginx  
Created symlink from /etc/systemd/system/multi‐user.target.wants/nginx.service to  /usr/lib/systemd/system/nginx.service.  
[root@lb02 conf.d]# systemctl start nginx

① Create a directory to store the four-layer load balancing configuration file

[root@lb02 ~]# vim /etc/nginx/nginx.conf  
events {          
    ....  
}  
include /etc/nginx/conf.c/*.conf;  
http {          
    .....  
} 
[root@lb02 ~]# mkdir /etc/nginx/conf.c

② Configure four-layer load balancing

[root@web03 conf.c]# cat lb_domain.conf   
stream {      
    upstream lb {              
        server 172.16.1.5:80 weight=5 max_fails=3 fail_timeout=30s;              
        server 172.16.1.6:80 weight=5 max_fails=3 fail_timeout=30s;      
    } 
    server {              
        listen 80;              
        proxy_connect_timeout 3s;              
        proxy_timeout 3s;              
        proxy_pass lb;      
    }  
}  
[root@web03 conf.c]# nginx ‐t  
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok 
nginx: configuration file /etc/nginx/nginx.conf test is successful  
[root@web03 conf.c]# systemctl reload nginx 
#配置本机hosts解析后浏览器访问并查看nginx日志

③Four-layer load balancing open log

#四层负载均衡是没有access的日志的，因为在nginx.conf的配置中，access的日志格式是配置在http下的，而四层负载均衡配置是在http以外的；

#如果需要日志则需要配置在stream下面  

[root@web03 conf.c]# cat lb_domain.conf   
stream {      
    log_format  proxy '$remote_addr $remote_port ‐ [$time_local] $status $protocol '                    '"$upstream_addr" "$upstream_bytes_sent" "$upstream_connect_time"' ;           access_log /var/log/nginx/proxy.log proxy;      
    upstream lb {              
        server 172.16.1.5:80 weight=5 max_fails=3 fail_timeout=30s;              
        server 172.16.1.6:80 weight=5 max_fails=3 fail_timeout=30s;      } 
    server { 
        listen 80;              
        proxy_connect_timeout 3s;              
        proxy_timeout 3s;              
        proxy_pass lb;      
    }  
}

5. Nginx Layer 4 load balancing port forwarding

①Use nginx four-layer load balancing to achieve tcp forwarding

请求负载均衡 5555    ‐‐‐>     172.16.1.7:22;  请求负载均衡 6666    ‐‐‐>     172.16.1.51:3306;

② Configure nginx four-layer load balancing to achieve tcp forwarding

[root@lb4‐01 ~]# cat /etc/nginx/conf.c/lb_domain.conf   
stream {      
    log_format  proxy '$remote_addr $remote_port ‐ [$time_local] $status $protocol '                        '"$upstream_addr" "$upstream_bytes_sent" "$upstream_connect_time"' ;       access_log /var/log/nginx/proxy.log proxy; 
    
#定义转发ssh的22端口      
 upstream ssh_7 {              
     server 10.0.0.7:22;      
    }  
    
#定义转发mysql的3306端口      
upstream mysql_51 {              
    server 10.0.0.51:3306;      
    }      
    server {              
        listen 5555;              
        proxy_connect_timeout 3s;              
        proxy_timeout 300s;              
        proxy_pass ssh_7;      
    } 
    server {              
        listen 6666;              
        proxy_connect_timeout 3s;              
        proxy_timeout 3s;              
        proxy_pass mysql_51;      
    }  
}