第八章：部署haproxy代理 搭建nginx集群
Introduction to Haproxy
Haproxy is an open source high-performance reverse proxy or one of the load balancing service software. It supports dual-machine hot backup, virtual host, TCP and HTTP-based application proxy and other functions.
Its configuration is simple, and it has a good health check function for server nodes (equivalent to keepalived health check). When the back-end server of its proxy fails, Haproxy will automatically
remove the failed server. When the server fails, it will recover. Finally, Haproxy will automatically add the RS server
Haproxy is particularly useful for those that have a lot of traffic. But it requires session maintenance or seven-layer application services. Haproxy runs on ordinary server hardware and can support tens of thousands of concurrent connections with only simple optimizations.
And its operating mode makes it easy and safe to integrate into the architecture of various websites, while preventing the application server from being exposed to the network.
Haproxy software introduces the functions of frontend and backend. Frontend (acl rule matching) allows operation and maintenance managers to make rule matching based on any HTTP request header, and then
direct the request to the relevant backend (server pools waiting for the front end to transfer the request to the server group) ). Through frontend and backup,
we can easily implement haproxy's 7-layer proxy function. haproxy is a rare and excellent proxy service software.
Haproxy supports two main proxy modes: the first is a 4-layer tcp proxy (for example: it can be used for mail service internal protocol communication server, Mysql service, etc.).
The second is a layer 7 proxy (like an HTTP proxy). In layer 4 tcp proxy mode, Haproxy only forwards traffic in both directions between client and server.
But in the 7-layer mode, Haproxy will analyze the application layer protocol and can control the protocol by running, rejecting, exchanging, adding, modifying or deleting the specified content in the request or response.
Introduction to haproxy main configuration file
vim /usr/src/haproxy-1.5.19/examples/haproxy.cfg
#Global configuration, used to set global parameters, is a process-level configuration, usually related to operating system configuration.
global
#Define global log, configured locally, output through local0, the default is info level, two
logs 127.0 can be configured. 0.1 local0 warning
#Define the log level [error warning info debug]
#log 127.0.0.1 local1 info
#Running path
chroot /usr/local/haproxy
#PID file storage path
pidfile /var/run/haproxy.pid
#Set the process of each haproxy process The maximum number of concurrent connections, which is equivalent to the command line option "-n"; the automatically calculated result of "ulimit -n" refers to this parameter setting.
maxconn 4096
#Run haproxy user, or use the keyword uid
user haproxy
#Run haproxy user group , or use the keyword gid
group haproxy
#Run haproxy
daemon in the background
#Set the number of haproxy processes to start, which can only be used for haproxy in daemon mode;
#Only one process is started by default. In view of various reasons such as difficulty in debugging, it is generally only used in Multi-process mode is used in scenarios where a single process can only open a few file descriptors.
nbproc 1
#Set the maximum number of file descriptors that can be opened by each process. By default, it will be automatically calculated, so it is not recommended to modify this option.
#ulimit-n 819200
#Debug level, generally only debug when a single process is started, and in the production environment Disabled.
#debug
#haproxy will not display any relevant information after starting, which is the same as adding the parameter "-q" when starting haproxy from the command line
#quiet
#Define the location where statistics are saved
stats socket /usr/local/haproxy/stats
# Default configuration
defaults
#Default mode [tcp: layer 4; http: layer 7; health: only returns OK]
mode http
#Inherit global log definition output
log global
#Log category, httplog
#option httplog
#If the back-end server needs to record For the client's real IP, you need to add the "X-Forwarded-For" field in the HTTP request; #But
when haproxy's own health detection mechanism accesses the back-end server, the access log should not be recorded. You can use except to exclude 127.0.0.0, that is haproxy itself.
#option forwardfor except 127.0.0.0/8
option forwardfor
#Enable the server-side shutdown function in the http protocol, and actively close the http channel after each request is completed, so that long connections are supported, the session can be reused, and each log record is All will be recorded.
option httpclose
#If an empty connection is generated, the log of this empty connection will not be recorded.
option dontlognull
#When the session with the backend server fails (server failure or other reasons), redistribute the session to other healthy servers on; when the failed server recovers, the session is directed to the recovered server;
#You can also use the "retries" keyword to set the number of connection attempts when determining session failure
option redispatch
retries 3
#When haproxy is under heavy load When high, links that have been processed in the current queue for a long time will be automatically terminated.
option abortonclose
#Default http request timeout
timeout http-request 10s
#Default queue timeout time, when the backend server is under high load, the request sent by haproxy will be put into In a queue.
timeout queue 1m
#The timeout time for the connection between haproxy and the back-end server.
timeout connect 5s
#After the client connects to haproxy, the data transmission is completed and there is no more data transmission, that is, the timeout time for the inactive connection.
timeout client 1m
# The timeout time for the inactive connection between haproxy and the back-end server.
timeout server 1m
#The default timeout time for new http request connection establishment. If the time is short, resources can be released as soon as possible to save resources.
timeout http-keep-alive 10s
#Heartbeat detection timeout
timeout check 10s
#Maximum number of concurrent connections
maxconn 2000
#Set the default load balancing method
#balance source
#balnace leastconn
#Statistics page configuration, the combination of frontend and backend, the name of the monitoring group can be customized
listen as needed admin_status
#Configure monitoring operation mode
mode http
#Configure statistics page access port
bind 0.0.0.0:1080
#Default maximum number of connections on the statistics page
maxconn 10
#http log format
option httplog
#Enable statistics
stats enable #Hide haproxy version information stats
on the statistics page
hide-version
#Monitoring page automatic refresh time
stats refresh 30s
#Statistics page access url
stats uri /stats
#Statistics page password box prompt text
stats realm mCloud\ Haproxy
#Monitoring page user and password: admin, multiple user names can be set
stats authadmin:admin
#Manually start/disable the back-end server, you can use the web management node
stats admin if TRUE
#Set the haproxy error page
errorfile 400 /usr/local/haproxy/errorfiles/400.http
errorfile 403 /usr/local/haproxy/errorfiles/403. http
errorfile 408 /usr/local/haproxy/errorfiles/408.http
errorfile 500 /usr/local/haproxy/errorfiles/500.http
errorfile 502 /usr/local/haproxy/errorfiles/502.http
errorfile 503 /usr/local/ haproxy/errorfiles/503.http
errorfile 504 /usr/local/haproxy/errorfiles/504.http
#Monitor the monitoring status of haproxy back-end server
listen site_status
bind 0.0.0.0:1081 #Listen port
mode http #http layer 7 mode
log 127.0.0.1 local2 err #[err warning info debug]
monitor-uri /site_status #Website health detection URL, used to check whether the website managed by HAProxy can be used. It returns 200 normally and 503 abnormally.
acl site_dead nbsrv(php_server) lt 1 #Define the strategy when the website is down. Return true when the number of effective machines in the specified backend hanging on the load balancer is less than 1 acl site_dead nbsrv(html_server) lt 1 acl site_dead nbsrv(backend_default
)
lt 1
monitor fail if site_dead #When the policy is met, 503 is returned. The online document says 500, but the actual test is 503
monitor-net 192.168.4.171/32 #Log information from 192.168.4.152 will not be recorded and forwarded
monitor-net 192.168.4.172/32
#frontend, name custom
frontend HAproxy_Cluster
#Define the front-end listening port, it is recommended to use the form of bind:80, otherwise there will be problems when making the cluster highly available, and the VIP will not be able to access when switching to other machines. bind
0.0.0.0 :80
#acl is followed by the rule name. When the requested url ends with .php, the match triggers the php_web rule. Both of the following writing methods are acceptable.
#When the requested url ends with .css, .jpg, .png, When .jpeg, .js, .gif ends, match and trigger static_web rules.
#acl static_web path_end .gif .png .jpg .css .js .jpeg
#acl static_web url_reg /.(css|jpg|png|jpeg|js| gif)$
#-i ignores case. When the requested host is a host starting with www.test.com, the dns_name rule is matched and triggered.
acl html_web hdr_beg(host) -i www.haproxytest.com
#acl html_web hdr_beg(host) 10.11.4.152
#When the client's IP is xxxx, match and trigger the src_ip rule.
#acl src_ip src xxxx
#If the acl rule php_web is matched, the request will be forwarded to the php_server group for processing; if the acl rule html_web is matched, the request will be forwarded to the html_server group. Processing.
use_backend php_server if php_web
use_backend html_server if html_web
#If none of the above rules match, transfer the request to the default_backend group for processing.
default_backend backend_default
#Backend backend configuration, configure the php_server group and html_server group
backend php_server
#Define the load balancing method as roundrobin method, that is, a weight-based round-robin scheduling algorithm, which is recommended when the server performance is relatively evenly distributed. #There are
also several load balancing methods as follows:
#-- static-rr: It is also a round-robin scheduling based on weight, but it is a static method , adjusting the backend unit weight at runtime will not use the new weight;
#-- source: perform hash operation based on the request source IP to match the backend server group;
#-- leastconn: Not suitable for environments with short sessions, such as http-based applications;
#-- uri: hash operation on the entire URI;
#-- uri_param: forward parameters in the URI;
#-- hdr() :Forwarding based on the http header. If there is no such header, roundrobin is used instead.
balance roundrobin
mode http
#Allow serverid to be inserted into the cookie.
Cookies can be defined after the serverid. SERVERID
#The heartbeat detection method is to detect the back-end server index.html file, and also There are other ways
option httpchk GET /index.html
#Backend server definition, maxconn 1024 represents the maximum number of connections to the server, cookie 1 represents serverid 1, weight represents weight (default 1, maximum 265, 0 means not to participate in the load Balance),
#check inter 1500 is to detect the heartbeat frequency, rise 2 is 2 correct times and the server is considered available, fall 3 is 3 failures and the server is unavailable.
server php1 192.168.4.171:80 maxconn 1024 cookie 1 weight 3 check inter 1500 rise 2 fall 3
backend html_server
balance source
mode http
server html1 192.168.4.172:80 maxconn 1024 cookie 1 weight 3 check inter 1500 rise 2 fall 3
backend backend_default
balance source
mode http
server default1 192.168.4.171:80 maxconn 1024 cookie 1 weight 3 check inter 1500 rise 2 fall 3
lab environment:
- linux-1 haproxy proxy server
192.168.10.1
2.linux-2 nginx-1
192.168.10.2
3.linux-3 nginx-2
192.168.10.3
experiment procedure:
1. Compile and install Nginx server (nginx-1, nginx-2)
1.nginx-1 server
[root@localhost ~]# yum -y install pcre-devel zlib-devel
[root@localhost ~]# tar xf nginx-1.12.0.tar.gz -C /usr/src/
[root@localhost ~]# cd /usr/src/nginx-1.12.0
[root@localhost ~]# useradd -M -s /sbin/nologin nginx
[root@localhost ~]# ./configure --prefix=/usr/local/nginx --user=nginx --group=nginx &&make &&make install
[root@localhost ~]# echo “
server 192.168.10.3
” >/usr/local/nginx/html/index.html[root@localhost ~]# /usr/local/nginx/sbin/nginx
[root@localhost ~]# netstat -anput |grep nginx
If you need to shut down the service /usr/local/nginx/sbin/nginx -s stop
2.nginx-2 server (you can use a script to install it and confirm that the source code software has been stored in the /root directory)
[root@localhost ~]# vim nginx-install.sh
#!/bin/bash
read -p "Please enter the content of the homepage:" index
yum -y install pcre-devel zlib-devel
tar zxvf nginx-1.12.0.tar.gz -C /usr/src/
cd /usr/src/nginx-1.12.0
useradd -M -s /sbin/nologin nginx
./configure --prefix=/usr/local/nginx --user=nginx --group=nginx && make && make install
echo “$index” > /usr/local/nginx/html/index.html
/usr/local/nginx/sbin/nginx
netstat -anput|grep nginx &>/dev/null
if [ $? -eq 0 ] ;then
echo "nginx service started successfully"
else
echo "nginx service did not start successfully, please check the installation process"
fi
…
[root@localhost ~]# chmod +x nginx-install.sh
[root@localhost ~]# ./nginx-install.sh
2. Haproxy proxy server
1. Compile and install haproxy
[root@localhost ~]# yum -y install pcre-devel bzip2-devel
[root@localhost ~]# tar xf haproxy-1.5.19.tar.gz -C /usr/src/
[root@localhost ~]# cd /usr/src/haproxy-1.5.19/
##64-bit system
[root@localhost ~]# make TARGET=linux26
[root@localhost ~]# make install
2.配置Haproxy
[root@localhost haproxy-1.5.19]# vim /usr/src/haproxy-1.5.19/examples/haproxy.cfg
k Note:
global --global part
log /dev/log local0 info
log /dev/log local1 notice ----Store the info and notice logs separately for viewing
maxconn 4096 ----Maximum number of connections
uid 99
gid 99 — User and group id
# pidfile /var/run/haproxy.pid —The path and file name of the pid file
daemon – running in the background
defaults ----Default configuration
log global ——Apply the global part of the log configuration
mode http ——The mode is http
option httplog
option dontlognull
retries 3 ---------Check the number of node failures. If it reaches 3 consecutive times, the node is considered unavailable. Use
maxconn 2000-----maximum number of connections
contimeout 5000-connection timeout 5000
clitimeout 50000
srvtimeout 50000----client and server timeout are both 50000
# option httpclose ----关闭客户端请求
listen webcluster 0.0.0.0:80 ----web cluster (listening address and interface)
option httpchk GET /index.html ----check http file
balance roundrobin - load balancing scheduling algorithm polling roundrobin
server inst1 192.168.10.2 :80 check inter 2000 fall 3
server inst2 192.168.10.3:80 check inter 2000 fall 3 - The address, name, port, check interval, and health check number of the server node are considered failed after 3 times
The log file in the /dev/log directory is a socket. It is the site of a communication line, and there is a data communication network between these end points (sockets).
Its communication process is:
the program accesses the socket address of the remote computer, and a communication line is established between the socket address of the accessed computer and the socket address of the remote computer.
[root@localhost ~]# cd
[root@localhost ~]# mkdir /etc/haproxy
Edit the configuration file as follows
[root@localhost ~]# vim /etc/haproxy/haproxy.cfg
global
log /dev/log local0 info
log /dev/log local1 notice
maxconn 4096
uid 99
gid 99
daemon
defaults
log global
mode http
option httplog
option dontlognull
retries 3
maxconn 2000
contimeout 5000
clitimeout 50000
srvtimeout 50000
listen webcluster 0.0.0.0:80
option httpchk GET /index.html
balance roundrobin
server inst1 192.168.10.2:80 check inter 2000 fall 3
server inst2 192.168.10.3:80 check inter 2000 fall 3
[root@localhost ~]# cp /usr/src/haproxy-1.5.19/examples/haproxy.init /etc/init.d/haproxy
[root@localhost ~]# chmod +x /etc/init.d/haproxy
[root@localhost ~]# ln -s /usr/local/sbin/haproxy /usr/sbin/haproxy
[root@localhost ~]# /etc/init.d/haproxy restart
3. Verification:
The client opens 2 IE browsers to visit: http://192.168.10.1
Check whether you can obtain different web content and it will be successful.
4. haproxy log management:
Haproxy's logs are output to the system's syslog by default, which is not very convenient to view. In order to facilitate the management of haproxy's logs, they are defined separately in the production environment.
[root@localhost ~]# cd /etc/rsyslog.d/
[root@localhost ~]# vim haproxy.conf
local0.* /var/log/haproxy/ha-info.log
local1.* /var/log/haproxy/ha-notice.log
#If you do not add the following configuration, in addition to writing logs in /var/log/haproxy.log, the message file will also be written.
[root@localhost ~]# vi /etc/sysconfig/rsyslog
Add to:
SYSLOGD_OPTIONS="-r -m 0 -c 2"
:wq
#-c 2 Use compatibility mode, the default is -c 5
#-r Enable remote logs
#-m 0 Mark timestamp. The unit is minutes. When it is 0, it means the function is disabled.
[root@localhost ~]# systemctl restart rsyslog
[root@localhost ~]# /etc/init.d/haproxy restart
[root@localhost ~]# vim /etc/haproxy/haproxy.cfg
global
log /dev/log local0 info
log /dev/log local1 notice
Upon checking the configuration haproxy.cfg has been configured
Verification test:
[root@localhost ~]# tail /var/log/haproxy/ha-info.log
Check whether any logs are generated
...
The relevant rsyslog configuration in the textbook is written in rainerscript language and is suitable for complex environments. Omitted - just use the above method to achieve
vim /etc/rsyslog.d/haproxy.conf
Edit content:
if ($programname == ‘haproxy’ and $syslogserverity-text == ‘info’) then -/var/log/haproxy/haproxy-info.log
& ~
if ($programname == ‘haproxy’ and $syslogserverity-text == ‘notice’) then -/var/log/haproxy/haproxy-notice.log
& ~
…
Haproxy parameter optimization
maxconn It is recommended to use 10240 daemon for the maximum number of connections.
The daemon process mode can use the non-daemon default
nbproc. It is recommended that the number of concurrent processes for load balancing be equal to or 2 times the current server cpu core.
The number of retries is to check the cluster nodes. If there are many nodes, it is concurrent. The amount is large, set to 2 to 3 times.
option http-server-close actively closes the http request option. Use this option in the production environment to avoid accumulation of http connections due to too long timeout time setting.
timeout http-keep-alive long connection timeout (10s) )
timeout http-request http request timeout time (5~10s) Increase the speed of http connection release
timeout client client timeout time