haproxy+nginx

Haproxy build a web cluster

Common Web cluster scheduler

目前常见的Web集群调度器分为软件和硬件，软件通常使用开源的LVS,Haproxy,Nginx,硬件一般使用比较多的是F5，也有很多人使用国内的一些
产品，如梭子，绿盟等

Haproxy Application Analysis

LVS in enterprise applications load capacity is very strong, but there is insufficient

LVS不支持正则处理，不能实现动静分离
对于大型网站，LVS的实施配置复杂，维护成本相对较高

Haproxy is a can provide high availability, load balancing, and proxy software is based on TCP and HTTP applications

特别适用于负载特别大的Web站点
运行在当前的硬件上可支持数以万计的并发连接连接请求

Haproxy of three algorithms

The RR (Round Robin)
the RR algorithm is the simplest and most commonly used method, round robin scheduling
Examples appreciated that
there are three nodes A, B, C will the first user is assigned access node A, the second user is assigned access to the node B, a third user may access node assigned to
a fourth user continued access node assigned to A, a polling access request to achieve load balancing assignment

The LC (Least the Connections )
the LC algorithm i.e. minimal connection algorithm, the number of connections size dynamic allocation request according to the node backend
Examples understanding
has three nodes A, B, C, the number of connections A respective nodes: 4, B: 5, C : 6, At this time, if a user with a first connection request will be assigned to the a, the number of connections of a: 5, B: 5, C: 6
second user, the request will be assigned to the a, the number of connections becomes a: 6, B: 5, C: 6, no new requests will be allocated to B, each time a new request is assigned to the connection section of the minimum number of customers
because the actual number of connections a, B, C will release dynamics, there will be very difficult to connect the same number of cases, so the algorithm compared rr algorithm has greatly improved, it is currently used in more of an algorithm

SH (the Source Hashing)
SH i.e. originated access scheduling algorithm, the algorithm used in some scenarios the Session session record in the server segment, and the like can be done based on the source cluster scheduling IP.Cookie
appreciated example
has three nodes A, B, C, a first user assigned to the first access a, a second user assigned to the first access B
when the first user a second visit will continue to be assigned to a, a second user a second visit when will still assigned to B, as long as the load balancing scheduler does not restart, the first user access will be assigned to a, the second user access is assigned to B, the scheduling cluster to achieve
this is to realize the benefits of scheduling algorithm to keep the session, but can cause load imbalance, some nodes to access large amounts of IP traffic when some very large, affecting business use

Three parts Haproxy profile

global: global configuration
defaults: the default configuration
listen: Application Component Configuration

global configuration parameters

log 127.0.0.1 local: 配置日志记录，配置日志记录，local0为日志设备，默认存放到系统日志
log 127.0.0.1 local notice: notice为日志级别，通常有24给级别
maxconn 4096 最连接数
uid :99 用户uid  d99: 用户gid

defaults configuration item to configure the default parameters, application components will generally be inherited, if not specifically stated in the application component, the default installation configuration parameter settings

log global :定义日志为global配置中的日志定义
mode http 模式为http
option httplog 采用http日志格式记录日志
retries 3 检查节点服务器失败连续达到三次则认为节点不可用
maxconn 2000 最大连接数
contimeout 5000 连接超时时间
clitimeout 50000 客户端超时时间
srvtimeout 50000 服务器超时时间

generally listen configuration item configuration application module parameters

listen appli4-backup 0.0.0.0:10004 定义一个appli4-backup的应用
option httpchk /index.html 检查服务器的index.html文件
option persist 强制将请求发送到已经down掉的服务器
balance roundrobin 负载均衡调度算法使用轮询算法
server inst1 192.168.100.201:80 check inter 2000 fall 3 定义在线节点
server inst2 192.168.100.202:80 check inter 2000 fall 3 定义备份节点

Here began the experiment:

We need three virtual machines

4 nginx server 192.168.100.201
. 5 Nginx server 192.168.100.202
. 7 Haproxy server 192.168.100.210

7Haproxy server, host-only mode configuration card

yum install bzip2-devel pcre-devel gcc gcc-c++ make -y

[root@localhost ~]# vim /etc/sysconfig/network-scripts/ifcfg-ens33 
BOOTPROTO=static
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=ens33
UUID=849aa04e-1874-490f-8cb0-b2fde4b9a6f8
DEVICE=ens33
ONBOOT=yes
IPADDR=192.168.100.210
NETMASK=255.255.255.0
GATEWAY=192.168.100.1

[root@localhost ~]# systemctl restart network
[root@localhost ~]# ifconfig

4nginx server, host-only mode configuration card

[root@localhost ~]# yum install pcre-devel zlib-devel gcc gcc-c++ -y ##安装环境包

BOOTPROTO=static
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes

IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=ens33
UUID=849aa04e-1874-490f-8cb0-b2fde4b9a6f8
DEVICE=ens33
ONBOOT=yes
IPADDR=192.168.100.201
NETMASK=255.255.255.0
GATEWAY=192.168.100.1
[root@localhost ~]# systemctl restart network
[root@localhost ~]# ifconfig

Server Configuration 4nginx

[root@localhost ~]# useradd -M -s /sbin/nologin nginx  ##创建程序性用户

[root@localhost ~]# mkdir /chen  ##创建挂载点
[root@localhost ~]# mount.cifs //192.168.100.23/LNMP /chen  ##挂载
Password for root@//192.168.100.23/LNMP:  

[root@localhost chen]# tar zxvf nginx-1.12.2.tar.gz -C /opt/  ##解压

[root@localhost chen]# cd /opt/
[root@localhost opt]# ls
nginx-1.12.2  rh
[root@localhost opt]# cd nginx-1.12.2/
[root@localhost nginx-1.12.2]# ls
auto     CHANGES.ru  configure  html     man     src
CHANGES  conf        contrib    LICENSE  README

./configure \
--prefix=/usr/local/nginx \
--user=nginx \
--group=nginx \
--with-http_stub_status_module

[root@localhost nginx-1.12.2]# make && make install ##编译

[root@localhost nginx-1.12.2]# ln -s /usr/local/nginx/sbin/nginx /usr/local/sbin/ ##做软链接让系统能识别nginx的所有人命令
[root@localhost nginx-1.12.2]# nginx -t  ##检查语法错误
nginx: the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok
nginx: configuration file /usr/local/nginx/conf/nginx.conf test is successful

Nginx write script in the system startup script convenient service Manager

[root@localhost nginx-1.12.2]# cd /etc/init.d/ ##到系统启动脚本

[root@localhost init.d]# vim nginx   ##写一个nginx脚本

#!/bin/bash
#chkconfig: - 99 20  #注释信息
#description: Nginx Service Control Script
PROG="/usr/local/nginx/sbin/nginx"  #这个变量，指向我的命令文件
PIDF="/usr/local/nginx/logs/nginx.pid"  #这个变量，指向nginx的进程号
case "$1" in
    start)
        $PROG                                              
        ;;
    stop)
        kill -s QUIT $(cat $PIDF) 
        ;;
    restart)                                                  
        $0 stop
        $0 start
        ;;
    reload)                                                  
        kill -s HUP $(cat $PIDF)
        ;;
    *)                                                           
                echo "Usage: $0 {start|stop|restart|reload}"
                exit 1
esac
exit 0

[root@localhost init.d]# chmod +x nginx  ##给Nginx提升权限
[root@localhost init.d]# chkconfig --add nginx  ##添加nginx
[root@localhost ~]# systemctl stop firewalld.service 
[root@localhost ~]# setenforce 0
[root@localhost ~]# cd /usr/local/nginx/html/
[root@localhost html]# echo "this is kgc web" > chen.html
[root@localhost ~]# systemctl restart nginx
[root@localhost ~]# netstat -ntap | grep ngixn
[root@localhost ~]# netstat -ntap | grep nginx
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN 

5nginx server, host-only mode configuration card

[root@localhost ~]# yum install pcre-devel zlib-devel gcc gcc-c++ -y ##安装环境包
BOOTPROTO=static
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes

IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=ens33
UUID=849aa04e-1874-490f-8cb0-b2fde4b9a6f8
DEVICE=ens33
ONBOOT=yes
IPADDR=192.168.100.202
NETMASK=255.255.255.0
GATEWAY=192.168.100.1
[root@localhost ~]# systemctl restart network
[root@localhost ~]# ifconfig

Server Configuration 5nginx

[root@localhost ~]# useradd -M -s /sbin/nologin nginx  ##创建程序性用户

[root@localhost ~]# mkdir /chen  ##创建挂载点
[root@localhost ~]# mount.cifs //192.168.100.23/LNMP /chen  ##挂载
Password for root@//192.168.100.23/LNMP:  

[root@localhost chen]# tar zxvf nginx-1.12.2.tar.gz -C /opt/  ##解压

[root@localhost chen]# cd /opt/
[root@localhost opt]# ls
nginx-1.12.2  rh
[root@localhost opt]# cd nginx-1.12.2/
[root@localhost nginx-1.12.2]# ls
auto     CHANGES.ru  configure  html     man     src
CHANGES  conf        contrib    LICENSE  README

./configure \
--prefix=/usr/local/nginx \
--user=nginx \
--group=nginx \
--with-http_stub_status_module

[root@localhost nginx-1.12.2]# make && make install ##编译

[root@localhost nginx-1.12.2]# ln -s /usr/local/nginx/sbin/nginx /usr/local/sbin/ ##做软链接让系统能识别nginx的所有人命令
[root@localhost nginx-1.12.2]# nginx -t  ##检查语法错误
nginx: the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok
nginx: configuration file /usr/local/nginx/conf/nginx.conf test is successful

Nginx write script in the system startup script convenient service Manager

[root@localhost nginx-1.12.2]# cd /etc/init.d/ ##到系统启动脚本

[root@localhost init.d]# vim nginx   ##写一个nginx脚本

#!/bin/bash
#chkconfig: - 99 20  #注释信息
#description: Nginx Service Control Script
PROG="/usr/local/nginx/sbin/nginx"  #这个变量，指向我的命令文件
PIDF="/usr/local/nginx/logs/nginx.pid"  #这个变量，指向nginx的进程号
case "$1" in
    start)
        $PROG                                              
        ;;
    stop)
        kill -s QUIT $(cat $PIDF) 
        ;;
    restart)                                                  
        $0 stop
        $0 start
        ;;
    reload)                                                  
        kill -s HUP $(cat $PIDF)
        ;;
    *)                                                           
                echo "Usage: $0 {start|stop|restart|reload}"
                exit 1
esac
exit 0

[root@localhost init.d]# chmod +x nginx  ##给Nginx提升权限
[root@localhost init.d]# chkconfig --add nginx  ##添加nginx
[root@localhost ~]# systemctl stop firewalld.service 
[root@localhost ~]# setenforce 0
[root@localhost ~]# cd /usr/local/nginx/html/
[root@localhost html]# echo "this is accp web" > chen.html
[root@localhost ~]# systemctl restart nginx
[root@localhost ~]# netstat -ntap | grep ngixn
[root@localhost ~]# netstat -ntap | grep nginx
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN 

Server Configuration 7haproxy

[root@localhost ~]# mount.cifs //192.168.100.25/LNMP chen/
Password for root@//192.168.100.25/LNMP:  
[root@localhost ~]# cd chen/
[root@localhost chen]# ls
Discuz_X3.4_SC_UTF8.zip  mysql-boost-5.7.20.tar.gz  php-7.1.10.tar.bz2
fang.png                 nginx-1.12.0.tar.gz        php-7.1.20.tar.gz
[root@localhost chen]# tar zxvf haproxy-1.5.19.tar.gz -C /opt/
[root@localhost ~]# cd /opt/
[root@localhost opt]# cd haproxy-1.5.19/
make TARGET=linux26 #编译
[root@localhost haproxy-1.5.19]# make install
[root@localhost haproxy-1.5.19]# mkdir /etc/haproxy
[root@localhost haproxy-1.5.19]# cp examples/haproxy.cfg /etc/haproxy/
[root@localhost haproxy-1.5.19]# vim /etc/haproxy/haproxy.cfg 

 21         redispatch #把这一行删掉，把请求给荡掉的服务器
 8         chroot /usr/share/haproxy #把这行删掉

 #先把25 行        srvtimeout      50000 后面的删掉100dd

 listen  webcluster 0.0.0.0:80
            option httpchk GET /chen.html
    balance roundrobin
    server inst1 192.168.100.201:80 check inter 2000 fall 3
        server inst2 192.168.100.202:80 check inter 2000 fall 3

The startup script in the script system

[root@localhost haproxy-1.5.19]# cp examples/haproxy.init /etc/init.d/haproxy
[root@localhost haproxy-1.5.19]# chmod +x /etc/init.d/haproxy 
[root@localhost haproxy-1.5.19]# chkconfig --add haproxy
[root@localhost haproxy-1.5.19]# ln -s /usr/local/sbin/haproxy /usr/sbin/haproxy
[root@localhost haproxy-1.5.19]# service haproxy start
Starting haproxy (via systemctl):                          [  确定  ]
[root@localhost haproxy-1.5.19]# systemctl stop firewalld.service 
[root@localhost haproxy-1.5.19]# setenforce 0

Client Test

The isolated and System logs

把原来的2dd
4         log /dev/log    local0 info
  5         log /dev/log    local0 notice

#创建一个空文件方便管理日志文件
[root@localhost haproxy]# cd /etc/rsyslog.d/
[root@localhost rsyslog.d]# vim haproxy.conf 

if ($programname == 'haproxy' and $syslogseverity-text == 'info')
then -/var/log/haproxy/haproxy-info.log
&~
if ($programname == 'haproxy' and $syslogseverity-text == 'notice')
then -/var/log/haproxy/haproxy-notice.log
&~
[root@localhost rsyslog.d]# systemctl restart rsyslog.service

Go to the client to refresh the page

7 server to view the logs have not been separated

[root@localhost rsyslog.d]# cd /var/log/
[root@localhost log]# cd haproxy/
[root@localhost haproxy]# ls
haproxy-info.log
[root@localhost haproxy]# cat haproxy-info.log 
Nov 29 16:23:01 localhost haproxy[18770]: 192.168.100.50:49418 [29/Nov/2019:16:23:01.526] webcluster webcluster/inst1 1/0/1/1/3 200 252 - - ---- 1/1/0/1/0 0/0 "GET /chen.html HTTP/1.1"