Detailed explanation of HAProxy installation and configuration

1. Introduction to haproxy

  HAProxy is an open source software developed by Frenchman Willy Tarreau. It is a high-performance TCP and HTTP load balancer that can handle more than 10,000 simultaneous client connections. Features are provided to provide cookie-based persistence, content-based switching, advanced traffic policing for overload protection, automatic failover, regular expression-based header control of runtime, web-based reporting, advanced logging to help Troubleshoot applications or network and other features.
  HAProxy provides high availability, load balancing, and proxying for TCP and HTTP applications, supporting virtual hosts. It is a free, fast and reliable load balancing solution. HAProxy is particularly suitable for web sites with heavy loads, which usually require session persistence or layer 7 processing. HAProxy runs on current hardware and can support tens of thousands of concurrent connections. And its operating mode makes it easy and secure to integrate into your current architecture, while protecting your web server from being exposed to the Internet.

Official website: https://www.haproxy.com/

1. Differences between LVS, Haproxy and Nginx

1. LVS implements soft load balancing based on the Linux operating system, while Haproxy and Nginx implement soft load balancing based on third-party applications.
2. LVS is an IP load balancing technology that can implement Layer 4 and cannot implement forwarding based on directories and URLs. Both Haproxy and Nginx can implement layer 4 and layer 7 technologies. Haproxy can provide comprehensive load balancing solutions for TCP and HTTP applications.
3. Because LVS works on the fourth layer of the ISO model, its status monitoring function is single, while Haproxy has richer and more powerful status monitoring functions and can support multiple status monitoring methods such as ports, URLs, and scripts.
4. Haproxy is powerful, but its overall performance is lower than LVS load balancing in layer 4 mode
5. Nginx is mainly used for web servers or cache servers
6. Haproxy also supports virtual hosts
7. Haproxy supports 8 load balancer strategies
8. The advantages of Haproxy can supplement some of the shortcomings of nginx, such as supporting Session retention, Cookie guidance, and supporting the detection of the status of the back-end server by obtaining the specified URL.
9. Haproxy is similar to LVS. It is just a load balancing software. In terms of efficiency, Haproxy has better load balancing speed than Nginx, and it is also better than Nginx in terms of concurrent processing.
10. Haproxy supports load balancing forwarding of TCP protocol

One picture summarizes the comparison.
A detailed comparison is made from the perspectives of the advantages and disadvantages, applicability and magnitude of the three load balancing software Nginx, LVS and HAProxy.

2. HAProxy features and advantages:

1. Supports original SSL, and supports both client and server SSL.
2. Supports IPv6 and UNIX sockets
3. Support HTTP Keep-Alive
4. Support HTTP/1.1 compression to save bandwidth
5. Support optimized health detection mechanism (SSL, scripted TCP, check agent...)
6. Supports layer 7 load balancing.
7. Reliability and stability are very good.
8. There are 40,000-50,000 concurrent connections, the maximum request processing per unit time is 20,000, and the maximum data processing is 10Gbps.
9. Supports 8 load balancing algorithms and supports session retention.
10. Support virtual hosting.
11. Supports connection rejection and fully transparent proxy.
12. Has a server status monitoring page.
13. Support ACL.

3. Three solutions for HAProxy to maintain sessions

HAProxy maintains sessions in order for the same client to access the server. There are three solutions: client IP, Cookie and Session.

1. Hash is calculated and saved through the client IP to ensure that when the same IP accesses the proxy server, it can be forwarded to the fixed real server.
2. Rely on the real server to send the client's cookie information for session maintenance.
3. The Session and server ID of the real server will be saved to achieve session persistence. (HAProxy only requires that the backend server can be connected to the network, and there is no cumbersome ARP configuration like LVS)

4. HAProxy's balance 8 load balancing algorithms:

1. roundrobin: Based on weighted round robin.
2. static-rr: Based on weighted round robin. Static algorithm, changes during runtime cannot take effect
3. source: Algorithm based on request source IP. Perform a hash operation on the source IP of the request, divide the result by the total weight of the backend server, and then forward it to a matching server. Enables client requests from the same IP to always be forwarded to a specific backend server.
4. leastconn: Minimum connection. (Suitable for database load balancing, not suitable for environments with short sessions)
5. uri: Perform a hash operation on part or the entire URI, divide it by the total weight of the server, and finally forward it to the matching backend.
6. uri_param: Forward according to the parameters in the URL path to ensure that the same user request is distributed to the same machine when the number of back-end servers remains unchanged.
7. hdr(<name>): Forward based on http header, if no http header exists. Then use simple round robin.

5. HAProxy main working mode

• TCP mode: A full-duplex connection will be established between the client and the server, and a simple mode will not perform any processing on Layer 7 messages.
  Usually used in SSL, SSH, SMTP and other application layers.
• http mode (general use): Client requests will be deeply analyzed before being forwarded to the backend server. All requests that are not compatible with the RFC format will be rejected.

2. HAProxy installation

There are two ways to install. Compared with compilation and deployment, the tar.gz package and yum methods are simpler and more convenient to install through yum.

1.HAProxy version description

This article will show how to quickly install HAProxy on CentOS 7.
But the disadvantage is that the default haproxy version in CentOS's yum source is relatively low.

Looking at the haproxy version that can be installed in our system, we can see that it is 1.5.18:

yum list haproxy

[root@localhost ~]# yum list haproxy
已加载插件：fastestmirror
Repodata is over 2 weeks old. Install yum-cron? Or run: yum makecache fast
Determining fastest mirrors
 * base: mirrors.bupt.edu.cn
 * extras: mirrors.bupt.edu.cn
 * updates: mirrors.huaweicloud.com
可安装的软件包
haproxy.x86_64                                                                          1.5.18-9.el7_9.1                                                                           update

What if we want to install the latest haproxy?

Method 1: rpm package installation process (recommended)

HAPROXY does not officially provide rpm-related packages. You can
download the rpm package from the third-party website through the rpm package of the third-party warehouse: https://pkgs.org/download/haproxy
. Select the corresponding version. Our demonstration is on CentOS7. 9 Install the newer version haproxy 2.2

1. Check the version of haproxy in the system yum: yum info haproxy
   The version information on CentOS7 is 1.5.18. The version is still very old.

   #CentOS7 上的版本信息 1.5.18  版本还是很老的了，我们在这个CentOS7.9 上演示编译安装最新版本的HAPROXY
   [root@localhost ~]# yum info haproxy
   已加载插件：fastestmirror
   Loading mirror speeds from cached hostfile
    * base: mirrors.huaweicloud.com
    * extras: mirrors.huaweicloud.com
    * updates: mirrors.huaweicloud.com
   可安装的软件包
   名称    ：haproxy
   架构    ：x86_64
   版本    ：1.5.18
   发布    ：9.el7_9.1
   大小    ：835 k
   源    ：updates/7/x86_64
   简介    ： TCP/HTTP proxy and load balancer for high availability environments
   网址    ：http://www.haproxy.org/
   协议    ： GPLv2+
   描述    ： HAProxy is a TCP/HTTP reverse proxy which is particularly suited for high
   

2. Download repo information filewget https://repo.ius.io/ius-release-el7.rpm

   [root@localhost ~]# wget https://repo.ius.io/ius-release-el7.rpm
   --2023-03-16 14:35:48--  https://repo.ius.io/ius-release-el7.rpm
   正在解析主机 repo.ius.io (repo.ius.io)... 104.85.67.135, 2600:140b:5000:98d::3898, 2600:140b:5000:982::3898
   正在连接 repo.ius.io (repo.ius.io)|104.85.67.135|:443... 已连接。
   已发出 HTTP 请求，正在等待回应... 200 OK
   长度：8412 (8.2K) [application/x-rpm]
   正在保存至: “ius-release-el7.rpm”
   100%[================================================================================================================================================>] 8,412       --.-K/s 用时 0s     
   [root@localhost ~]# ll
   -rw-r--r--. 1 root root    8412 1月   3 14:05 ius-release-el7.rpm
   

3. Install epel-releaseyum -y install epel-release

4. Install ius-release-el7.rpmrpm -Uvh ius-release*rpm

5. Install haproxy version 2.2yum install haproxy22

6. Start haproxysystemctl start haproxy

# 安装 haproxy 2.2.22版
[root@localhost ~]# yum install haproxy22
# 查看端口监听 看不到有  haproxy 相关的
[root@localhost ~]# ss -tlnp | grep haproxy   

# 启动服务，再查看端口监听
[root@localhost ~]# systemctl start haproxy
[root@localhost ~]# ss -tlnp | grep haproxy   
LISTEN     0      128          *:5000                     *:*                   users:(("haproxy",pid=1416,fd=7))

# 查看版本
[root@localhost ~]# haproxy -v
HA-Proxy version 2.2.29-c5b927c 2023/02/14 - https://haproxy.org/
Status: long-term supported branch - will stop receiving fixes around Q2 2025.
Known bugs: http://www.haproxy.org/bugs/bugs-2.2.29.html
Running on: Linux 3.10.0-1127.el7.x86_64 #1 SMP Tue Mar 31 23:36:51 UTC 2020 x86_64
# 查看生成的文件   
[root@localhost ~]# rpm -ql haproxy22    
/etc/haproxy     					#haproxy 主要目录之一
/etc/haproxy/haproxy.cfg			#此文件是配置文件，很重要
/etc/logrotate.d/haproxy
/etc/sysconfig/haproxy
/usr/bin/halog
/usr/bin/iprange
/usr/lib/systemd/system/haproxy.service		#haproxy 启动文件
/usr/sbin/haproxy
/usr/share/doc/haproxy22-2.2.29
/usr/share/doc/haproxy22-2.2.29/51Degrees-device-detection.txt
………

Method 2: IUS installation process

IUS is a yum repository that provides newer versions of select software for RHEL and CentOS.

It provides yum sources of the latest software versions for CentOS by RHEL.

1. Configure IUS yum source:vim /etc/yum.repos.d/ius-7.repo

   [ius]
   name = IUS for Enterprise Linux 7 - $basearch
   baseurl = https://repo.ius.io/7/$basearch/
   enabled = 1
   repo_gpgcheck = 0
   gpgcheck = 1
   gpgkey = https://repo.ius.io/RPM-GPG-KEY-IUS-7
   [ius-debuginfo]
   name = IUS for Enterprise Linux 7 - $basearch - Debug
   baseurl = https://repo.ius.io/7/$basearch/debug/
   enabled = 0
   repo_gpgcheck = 0
   gpgcheck = 1
   gpgkey = https://repo.ius.io/RPM-GPG-KEY-IUS-7
   [ius-source]
   name = IUS for Enterprise Linux 7 - Source
   baseurl = https://repo.ius.io/7/src/
   enabled = 0
   repo_gpgcheck = 0
   gpgcheck = 1
   gpgkey = https://repo.ius.io/RPM-GPG-KEY-IUS-7
   

2. Clear cache:yum clean all

3. refresh cache:yum makecache fast

4. Check the version: yum list|grep haproxy
   You can see that the installable version under the base source is 1.5.18. There are many versions under the ius source, including the latest 2.2.

   [root@localhost ~]# yum list|grep haproxy
   haproxy.x86_64                              1.5.18-9.el7_9.1           updates  
   haproxy18u.x86_64                           1.8.30-1.el7.ius           ius      
   haproxy20.x86_64                            2.0.30-1.el7.ius           ius      
   haproxy22.x86_64                            2.2.29-1.el7.ius           ius      
   pcp-pmda-haproxy.x86_64                     4.3.2-13.el7_9             updates  
   [root@localhost ~]# 
   

5. Install version 2.2:yum install -y haproxy22

6. View installed haproxy:yum list installed |grep haproxy

7. View version: haproxy -v, you can see that the version is 2.2.1

8. View haproxy status: systemctl status haproxy.
   Because we have not started haproxy yet, the status printed now is inactive.

3. Use of HAProxy

1.HAProxy configuration file

The haproxy configuration file consists of two parts: global settings and proxy settings, divided into five sections: global, defaults, frontend, backend, listen

• global: Global configuration, mainly used to define global parameters, is a process-level configuration, usually related to operating system configuration.
• default: Configure default parameters, which can be used in frontend, backend, and Listen components.
• frontend: The front-end virtual node that receives the request. The frontend can specify the backend to specifically use the backend.
• backend: Configuration of backend service cluster, real server, one backend corresponds to one or more entity servers.
• listen: A combination of fronted and backend, such as the haproxy instance status monitoring part configuration. The only configuration method before Haproxy1.3.

Configuration file pathvim /etc/haproxy/haproxy.cfg

Global global configuration

global
    log 127.0.0.1 local0  info		//定义haproxy日志输出设置
    log loghost local0 info		//定义haproxy日志级别
    maxconn 20480			//定义最大连接数
    chroot /usr/local/haproxy		//chroot运行路径
    pidfile /var/run/haproxy.pid	//haproxy进程PID文件   
    user haproxy			//运行haproxy用户，可用uid代替
    group haproxy			//运行haproxy用户组，可用gid代替
    daemon				//以后台形式运行haproxy

defaults configuration

• Used to set configuration default parameters, which can be used in frontend, backend, and listen components. The parameter values ​​set in this section will be automatically referenced to the frontend, backend, and listen sections below by default. If some parameters belong to public configuration, you only need to add them once in the defaults section. And if the frontend, backend, and listen parts are also configured with the same parameters as the defaults part, the values ​​corresponding to the defaults part parameters are automatically overwritten.
• mode http: Set the running mode of haproxy. There are three {http|tcp|health}. Note: If layer 4 applications (mode tcp) are also used in haproxy, it is not recommended to define the operating mode of haproxy here.
  tcp模式: In this mode, the client and server will establish a full-duplex connection without any inspection of Layer 7 messages. The default is tcp mode, which is often used in SSL, SSH, SMTP and other applications.
  http模式: In this mode, client requests will be deeply split before being forwarded to the backend server, and all requests that are not compatible with the RFC format will be rejected.
  health: Basically no longer used.

defaults
    mode http		//所处理的类别（7层代理http，4层代理tcp）
    log global		//引入global定义的日志格式
    option dontlognull	//不记录健康检查日志信息
    option httpclose	//每次请求完毕后主动关闭http通道,haproxy不支持keep-alive模式
    option httplog	//日志类别为http日志格式
    option forwardfor	//如果后端服务器需要获取客户端的真是ip，需要配置的参数，可以从http header中获取客户端的ip
    option redispatch
    balance roundrobin	//设置默认负载均衡方式，轮询方式
    timeout connect 10s	//默认连接超时时间
    timeout client 10s	//默认客户端超时时间
    timeout server 10s	//默认服务器超时时间
    timeout check 10s	//设置超时检查超时时间 
    maxconn 60000	//最大连接数
    retries 3		//3次连接失败就认为服务器不可用，也可以通过后面设置

frontend configuration

• frontend is a component introduced after haproxy version 1.3, along with the backend component. By introducing these components, the complexity of haproxy configuration files is simplified to a great extent. forntend can directly specify the backend to be used based on ACL rules.
  like:

frontend http_80_in 
bind 0.0.0.0:80    //设置侦听端口，即haproxy提供的web服务端口，和lvs的vip类似
mode http	
log  global   
    option httpclose  
    option httplog  
option forwardfor
default_backend  webserver //设置请求默认转发的后端服务池

backend configuration
is used to define the configuration of the backend service cluster, real server, one Backend corresponds to one or more entity servers

backend  webserver  //定义webserver服务器组
mode  http
option  redispath
option  abortonclose
balancer  source   //负载均衡的方式，源哈希算法
cookie  SERVERID  //允许插入serverid到cookie中，serverid后面可以定义
option  httpdchk  GET  /test.html    //心跳测试
server  web1  192.168.92.100:80  cookie  2 weight 3  check  inter 2000  rise 2 fall 3 maconn 8

The listen configuration
is often used for status page monitoring and backend server inspection. It is a combination of Fronted and backend.

listen admin_stats	//frontend和backend的组合体，监控组的名称，按需自定义名称
    bind 0.0.0.0:8189	//侦听端口
    stats enable	//开启监控
    mode http
    log global
    stats uri /haproxy_stats    //监控页面的url访问路径
    stats realm Haproxy\ Statistics  //监控页面的提示信息
    stats auth admin:admin  //监控页面的用户和密码
    #stats hide-version   //隐藏统计页面上的haproxy版本信息
    stats admin if TRUE  //手工启用/禁用，后端服务器haproxy
    stats refresh 30s   //每个30秒自动刷新监控页面

Access the monitoring interface:http://192.168.92.110:8189/haproxy_stats

2. Implement a load balancing

Environmental preparation

CPU name	IP address	Remark
Haproxy	192.168.92.110	haproxy
web server	192.168.92.100	web
web server	192.168.92.1101	web

listen implements a simple load balancing

#实现一个负载均衡
listen  webcluster 0.0.0.0:80
    option  httpchk GET /index.html
    balance roundrobin
    server  inst1 192.168.92.101:80 check inter 2000 fall 3
    server  inst1 192.168.92.100:80 check inter 2000 fall 3

Restart Haproxy after modifying the configuration file
systemctl restart haproxy

Configure ACL to handle different URL request ACL:

Access Control Lists (ACL, Access Control Lists) is an access control technology based on packet filtering. It can filter data packets transmitted through the server according to set conditions (condition matching), that is, it can filter the received packets. Matching and filtering, based on the source address, source port, target address, target port, request method, URL, file suffix and other information content in the request message header, match and perform further operations, such as allowing it to pass or discarding it.
For specific other usage methods, please refer to: Haproxy ACL configuration and cases

The following commonly used URL matching is demonstrated as an example, which is similar to the location matching path in nginx. The
configuration syntax and parameters of ACL in Haproxy:

acl语法：
acl    <aclname>      <criterion>  [flags]  [operator]  [<value>]
acl 该条acl规则的名称    匹配规范   匹配模式  具体操作符  操作对象类型

frontend http-in
    bind 0.0.0.0:80
    acl web url_beg  /web  #配置路基url 定义为web
    use_backend admin_web if web #如果满足web条件 走admin_web 服务
    default_backend admin_server #默认服务

backend admin_web #定义web服务
    mode http
    balance roundrobin  # Load Balancing algorithm
    option httpchk
    option forwardfor
    server  web1 192.168.92.101:80 check inter 2000 fall 3

backend admin_server #定义后端服务
    mode http
    balance roundrobin  # Load Balancing algorithm
    option httpchk
    option forwardfor
    server  web2 192.168.92.100:80 check inter 2000 fall 3

Parse configuration:

1. When the accessed URL contains web fields, it will go to the 192.168.92.101 server
2. Others go to the 192.168.92.100 server by default.
   

3. Start problem handling

After modifying the configuration file, there may be an error message that
haproxy cannot start. The error message is:
Starting proxy admin_stats: cannot bind socket [0.0.0.0:8189]

. Solution:
setsebool -P haproxy_connect_any=1
Restart it and it will be up.
systemctl start haproxy

[root@bogon ~]# setsebool -P haproxy_connect_any=1
[root@bogon ~]# systemctl start haproxy

4. Log processing

Haproxy's logs are output to the system's syslog by default. In order to better manage Haproxy's logs, they are generally defined separately in the production environment.

1.Haproxy log configuration

[root@localhost ~]# vim /etc/haproxy/haproxy.cfg 
# this config needs haproxy-1.1.28 or haproxy-1.2.1
 
global
    #log 127.0.0.1  local0 
    #log 127.0.0.1  local1 notice
     
    log /dev/log    local0 info
    log /dev/log    local0 notice
    
[root@localhost ~]# systemctl restart haproxy

These two lines of configuration are placed in the global option, mainly to record Haproxy's info and notice logs into different log files.

2. Modify rsyslog configuration

In order to facilitate management, Haproxy-related configurations are defined independently haproxy.confand placed /etc/rsyslog.d/under . When rsyslog starts, it will automatically load all configuration files in this directory.
vim /etc/rsyslog.d/haproxy.conf

[root@localhost ~]# vim /etc/rsyslog.d/haproxy.conf
if ($programname == 'haproxy' and $syslogserverity-text == 'info') then -/var/log/haproxy/haproxy-info.log &~
if ($programname == 'haproxy' and $syslogserverity-text == 'notice') then -/var/log/haproxy/haproxy-notice.log &~　　

Record haproxy's info log to /var/log/haproxy/haproxy-info.log,
and record notice log to /var/log/haproxy/haproxy-notice.log.

&~Indicates that after writing to the log file, rsyslog stops processing this information, (rainerscript script language)

3. Restart the rsyslog service and check whether the log file is created successfully.

[root@localhost ~]# systemctl restart rsyslog
#查看日志文件是否创建成功
[root@localhost ~]# ls -l /var/log/haproxy/haproxy-info.log
[root@localhost ~]# ls -l /var/log/haproxy/haproxy-notice.log