06: keepalive high availability cluster (new)

1.1 keepalived High Availability Software

Reference:
 HAProxy: 
     https://blog.csdn.net/l835311324/article/details/83031084
    HTTPS://blog.csdn.net/qq_41772936/article/details/80718014
    https://blog.51cto.com/3381847248/ 1977073

keepalive: 
    https://blog.csdn.net/celeste7777/article/details/49096167
    https://v3u.cn/a_id_117

　　1, keepalived-- monitoring inspection

　　　　　　Note: keepalive software has two functions: monitoring inspection, VRRP redundancy protocol

　　　　　　1. keepalive web server role is to detect a state, if a web server is down, the fault will be removed from the cluster machine

　　　　　　2. keepalieve will check the cluster service is normal from the following three:

　　　　　　　　1) Layer3: by ICMP protocol ping test

　　　　　　　　2) layer4: such as web services, keepalived 80 checks whether to activate port

　　　　　　　　3) Layer7: According to the setting server checks whether the user is running normally

　　2, keepalived - VRRP redundancy protocol principle

　　　　　　Note: vrrp virtual routing redundancy protocol, is the single point of failure problem when a mechanism to determine vip way through the campaign

　　　　　　1. Keepalived availability vrrp protocol communication is adopted, it is determined vrrp standby mechanism through the campaign

　　　　　　2. Keepalived primary server will always send VRRP broadcast packets, prepared to tell it alive

　　　　　　3. When the backup machine monitor broadcast packets sent by the master, artificially master is unavailable, all prepared by the machine configuration file priority election of a new master

　　　　　　4. The new host will start to take over the resource related services, ensure business continuity

1.2 keepalive + haproxy High Availability Cluster

　　1, real keepalive cluster deployment architecture diagram

　　　　1) haproxy configuration

　　　　　　　　1. haproxy frontend module is configured to be responsible for the interface to a user request, backend definition can define a set of back-end servers (in fact keepalive host group)

　　　　　　　　2. a frontend can have multiple backend, backend also allows a number of different frontend call ( backend api interface can be invoked multiple front-end )

　　　　　　　　3. The specific process is that when a user request is received frontend, to the processing request specified by the backend our configuration.

　　　　　　　　4. backend then distribute requests to the real back-end server

　　　　2) keepalive configuration

　　　　　　　　1. Two keepalive servers while monitoring a virtual IP, the same time only keepalive-master cluster capable of service agent

　　　　　　　　2. When the keepalive-master is down, keepalive-slave master will immediately replace the cluster and agency services

1.3 Use docker-compose arranged keepalive

　　1, Environment Description (Borrowing compose test)

'' 'Environment, a virtual machine only, and then start using the docker-compose three analog docker container HAProxy, Keepalive-Master, Slave-Keepalive' ''
 # HAProxy: 0.0.0.0:8000 (docker-compose the network mapped to the host port 8000 HAProxy)
# Keepalive-master: 172.20.128.2 (the VIP: 172.20.128.4: 80)
# Keepalive-Slave: 172.20.128.3  (the VIP: 172.20.128.4: 80)

　　2, deployment keepalive project code

version: "3"
services:
  nginx_master:
    build:
      context: ./
      dockerfile: ./Dockerfile
    volumes:
      - ./index-master.html:/usr/share/nginx/html/index.html
      - ./favicon.ico:/usr/share/nginx/html/favicon.ico
      - ./keepalived-master.conf:/etc/keepalived/keepalived.conf
    networks:
      static-network:
        ipv4_address: 172.20.128.2
    cap_add: 
      - NET_ADMIN
  nginx_slave:
    build:
      context: ./
      dockerfile: ./Dockerfile
    volumes:
      - ./index-slave.html:/usr/share/nginx/html/index.html
      - ./favicon.ico:/usr/share/nginx/html/favicon.ico
      - ./keepalived-slave.conf:/etc/keepalived/keepalived.conf
    networks:
      static-network:
        ipv4_address: 172.20.128.3
    cap_add: 
        - NET_ADMIN
  proxy:
    image: haproxy:1.7-alpine
    ports:
      - 8000:6301
    volumes:
      - ./haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg
    networks:
      - static-network

networks:
  static-network:
    ipam:
      config:
        - subnet: 172.20.0.0/16

Docker-compose.yml

# 使用的基础镜像源(alpine是精简的Linux内核)
FROM nginx:1.13.5-alpine
# 更新最新本地镜像源  && 升级软件
RUN apk update && apk upgrade
# 安装keepalived等相关软件
RUN apk add --no-cache bash curl ipvsadm iproute2 openrc keepalived && \
    rm -f /var/cache/apk/* /tmp/*
# 将nginx和keepalive启动脚本从宿主机拷贝到docker容器中
COPY entrypoint.sh /entrypoint.sh
# 给启动脚本添加权限
RUN chmod +x /entrypoint.sh
# 启动服务
CMD ["/entrypoint.sh"]

Dockerfile 构建keepalive镜像的dockerfile

# vrrp_script区域主要用来做健康检查的，当时检查失败时会将vrrp_instance的priority减少相应的值
vrrp_script chk_nginx {
    script "pidof nginx"
    interval 2
}

# 设置外网的VIP(虚拟ip)
vrrp_instance VI_1 {
    # 可以是MASTER或BACKUP，不过当其他节点keepalived启动时会将priority比较大的节点选举为MASTER
    state MASTER
    # 对外提供服务的网络接口，用来发VRRP包
    interface eth0
    # 取值在0-255之间，用来区分多个instance的VRRP组播， 同一网段中该值不能重复，并且同一个vrrp实例使用唯一的标识
    virtual_router_id 33
    # 用来选举master的，要成为master，那么这个选项的值最好高于其他机器50个点，该项取值范围是1-255（在此范围之外会被识别成默认值100）
    priority 200
    # 发VRRP包的时间间隔，即多久进行一次master选举，可以认为是健康查检时间间隔，单位为秒
    advert_int 1
    # 表示发送VRRP单播报文使用的源IP地址，当前keepalive真实ip
    unicast_src_ip 172.20.128.2
    # 表示对端接收VRRP单播报文的IP地址（防止上游交换机禁用广播）
    unicast_peer {
        172.20.128.3
    }
    
    authentication {
        # 定义认证类型
        auth_type PASS
        # 认证密码  这些相当于暗号
        auth_pass letmein
    }
    # keepalive对外的虚拟地址，访问虚拟地址会访问keepalive的master地址
    virtual_ipaddress {
        172.20.128.4/24 dev eth0
    }
    # track_script用于追踪脚本，用以实现某个检测功能
    track_script {
        chk_nginx
    }
}

# 配置keepalive
virtual_server 172.20.128.2 80 {
    delay_loop 15
    lb_algo rr
    lb_kind DR
    nat_mask 255.255.255.0
    persistence_timeout 50
    protocol TCP

    # keepalive代理的真实服务ip
    real_server 172.16.16.177 80 {
         weight 1
        TCP_CHECK {
            connect_port 80
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 4
        }
    }
    # keepalive代理的真实服务ip
    real_server 172.16.16.178 80 {
        weight 1
        TCP_CHECK {
            connect_port 80
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 4

        }
    }
}

keepalived-master.conf

vrrp_script chk_nginx {
    script "pidof nginx"
    interval 2
}

vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 33
    priority 100
    advert_int 1
    unicast_src_ip 172.20.128.3
    unicast_peer {
        172.20.128.2
    }
    
    authentication {
        auth_type PASS
        auth_pass letmein
    }
    
    virtual_ipaddress {
        172.20.128.4/24 dev eth0
    }
    
    track_script {
        chk_nginx
    }
}

keepalived-slave.conf

# global常用配置项
global
    # haproxy默认没有记录日志的文件，需要依赖rsyslog收集，具体方法，首先在配置文件的global段添加一条配置项
    log 127.0.0.1 local0
    # 最大并发连接数；默认为2000
    maxconn 4096
    daemon
    # 要启动的haproxy的进程数量；默认为1个，推荐为1个
    nbproc 4

defaults
    log 127.0.0.1 local3
    mode http
    option dontlognull
    option redispatch
    retries 2
    maxconn 2000
    balance roundrobin
    timeout connect 5000ms
    timeout client 5000ms
    timeout server 5000ms

# frontend部署前端服务，负责给用户请求一个接口
frontend main
    # 监听本docker容器中所有访问6301端口的请求
    # 监听端口，即haproxy提供web服务的端口，和lvs的vip端口类似
    bind *:6301
    # 定义一个服务器组 http_back
    default_backend webserver

# backend可以定义后端一组服务器
backend webserver
    server ngxin_master 172.20.128.4:80 check inter 2000 rise 2 fall 5
# 具体流程就是当frontend接受到用户请求，根据我们的配置交给指定的backend处理请求，backend再将请求分发到真正的后端服务器

haproxy.cfg

#!/bin/sh

/usr/sbin/keepalived -n -l -D -f /etc/keepalived/keepalived.conf --dont-fork --log-console &

nginx -g "daemon off;"

entrypoint.sh keepalive启动脚本

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <meta http-equiv="X-UA-Compatible" content="ie=edge">
    <title>主机</title>
<style>

#box{ 
    margin: 0px auto;
    font-family: 'Times New Roman', Times, serif; 
    font-size: 30px;
    font-style: initial;
    color: aliceblue;
    }

body{
    background-color: black
}

</style>
</head>
<body>

    <div id="box">

        主机

    </div>
    
</body>
</html>

index-master.html

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <meta http-equiv="X-UA-Compatible" content="ie=edge">
    <title>从机</title>
<style>

#box{ 
    margin: 0px auto;
    font-family: 'Times New Roman', Times, serif; 
    font-size: 30px;
    font-style: initial;
    color: aliceblue;
    }

body{
    background-color: black
}

</style>
</head>
<body>

    <div id="box">

        从机

    </div>
    
</body>
</html>

index-slave.html

　　3、测试keepalive

　　　　1）使用docker-compose开启keepalive集群

[root@linux-node4 nginx_keepalive]# docker-compose up       # 使用docker-compose部署keepalive（是在交互窗口运行的）
[root@linux-node4 nginx_keepalive]# docker ps               # 部署完成后新打开一个终端查看运行状态
CONTAINER ID        IMAGE                         PORTS                    NAMES
8fc6f148f02d        nginx_keepalive_nginx_master  80/tcp                   nginx_keepalive_nginx_master_1
5c94af00a93e        nginx_keepalive_nginx_slave   80/tcp                   nginx_keepalive_nginx_slave_1
93afe48d9b86        haproxy:1.7-alpine            0.0.0.0:8000->6301/tcp   nginx_keepalive_proxy_1

　　　　2）关闭keepalive-master模拟机器故障测试自动切换

[root@linux-node4 compose_lnmp]# docker stop nginx_keepalive_nginx_master_1     # 关闭keepalive-master模拟机器故障
[root@linux-node4 compose_lnmp]# docker ps                                      # 取到keepalive-master已经关闭
CONTAINER ID        IMAGE                        PORTS                    NAMES
c21d0edda6d3        nginx_keepalive_nginx_slave  80/tcp                   nginx_keepalive_nginx_slave_1
ab283ebcfaf1        haproxy:1.7-alpine           0.0.0.0:8000->6301/tcp   nginx_keepalive_proxy_1
[root@linux-node4 nginx_keepalive]# docker rm -f $( docker ps -a | grep keepalive |awk '{print $1}')   # 如果要停用最好删除创建的docker镜像

　　　　3）项目地址

　　　　　　　　https://gitee.com/xiaonq110/nginx_keepalive.git