Service registration and discovery are indispensable and important components in the microservice architecture. At first, services were all single-node, which did not guarantee high availability and did not consider the pressure bearing of the service. Calls between services were simply accessed through interfaces. Until the emergence of distributed architecture with multiple nodes, the initial solution was to load balance the service front-end. In this way, the front-end must know the network location of all back-end services and configure them in the configuration file. There will be several problems here

If you need to call the backend service AN, you need to configure the network locations of N services, which is very troublesome.
Changes in the network location of the backend service require changes to the configuration of each caller.
Since there are these problems, service registration and discovery are the solutions to these problems. The back-end service AN can register its current network location to the service discovery module, and the service discovery is recorded in the form of KV. K is generally the service name, and V is IP:PORT. The service discovery module performs health checks regularly and polls to see whether these backend services can be accessed. When the front-end calls the back-end service AN, it goes to the service discovery module to ask for their network location, and then calls their services. This method can solve the above problem. The front end does not need to record the network location of these back-end services at all. The front-end and back-end are completely decoupled!

2. Docker-consul cluster

1.Docker-consul

Consul is Google's open source service management software developed using the Go language. Supports multi-data centers, distributed high availability, service discovery and configuration sharing. The Raft algorithm is used to ensure high availability of services. It has built-in service registration and discovery framework, distribution consistency protocol implementation, health check, Key/Value storage, and multi-data center solution, and no longer needs to rely on other tools (such as ZooKeeper, etc.).

Service deployment is simple, with only a runnable binary package. Each node needs to run an agent, which has two operating modes: server and client. It is officially recommended that each data center requires 3 or 5 server nodes to ensure data security and ensure that the server-leader election can be carried out correctly.

In client mode, all services registered to the current node will be forwarded to the server node, and this information will not be persisted.

In server mode, the function is similar to that of client mode. The only difference is that it will persist all information locally, so that in the event of a failure, the information can be retained. The server-leader is the boss of all server nodes. It is different from other server nodes in that it needs to be responsible for synchronizing registered information to other server nodes. It is also responsible for the health monitoring of each node.

Some key features provided by consul

Service registration and discovery: Consul makes service registration and service discovery easy through DNS or HTTP interfaces. Some external services, such as those provided by saas, can also be registered in the same way.

Health check: Health check allows Consul to quickly alert on operations in the cluster. Integration with service discovery can prevent services from being forwarded to failed services.

Key/Value storage: A system used to store dynamic configurations. Provides a simple HTTP interface that can be operated anywhere.

Multi-datacenter: Support any number of regions without complex configuration.

Installing consul is for service registration, that is, some information about the container itself is registered in consul, and other programs can obtain the registered service information through consul. This is service registration and discovery.

2. recorder

Gliderlabs/Registrator can check the running status of the container and automatically register it, and can also log out the service of the docker container to the service configuration center. Currently Consul, Etcd and SkyDNS2 are supported.

3.Consul-template

Consul-Template is an application that automatically replaces configuration files based on Consul. Consul-Template is a daemon process used to query Consul cluster information in real time, update any number of specified templates on the file system, and generate configuration files. After the update is completed, you can choose to run a shell command to perform the update operation, and then reload the service configuration.

Consul-Template can query the service directory, Key, Key-values, etc. in Consul. This powerful abstraction and query language template makes Consul-Template particularly suitable for dynamically creating configuration files. For example: Create Apache/Nginx Proxy Balancers, Haproxy Backends, etc.

3. Docker-consul implementation process

Take configuring nginx load balancing as an example

First configure consul-agent. There are two modes: server and client.

consul_client collects automatically discovered information, forwards all information that needs to be registered to the server node, and does not persist this information.
consul_server persists all information locally, synchronizes information to other server nodes through server-leader, and monitors the health of each node.

Then discover the network location of the application through the registrator and send it to the automatic discovery module of consul agent for registration.

consul-template automatically replaces the service configuration file based on consul's registered information (needs to write a template)

4. Docker-consul cluster configuration

Download consul service

#解压软件后移动到/usr/local/bin/下
mv consul /usr/local/bin/
#创建数据目录，启动服务
mkdir /var/lib/consul_data
consul agent \
-server \
-bootstrap \
-ui \
-data-dir=/var/lib/consul_data \
-bind=192.168.116.70 \
-client=0.0.0.0 \
-node=consul-server01 &> /var/log/consul.log &

Common startup options

Options	effect
-server	Start as server. The default is client
-bootstrap	Used to control whether a server is in bootstrap mode. There can only be one server in bootstrap mode in a data center. When a server is in bootstrap mode, it can elect itself as the server-leader.
-bootstrap-expect=2	The minimum number of servers required by the cluster. When it is lower than this number, the cluster will fail.
-ui	Specify to open the UI interface, so that you can access consul's own web UI interface through an address such as http://localhost:8500/ui.
-data-dir	Specify the data storage directory.
-bind	Specify the communication address used within the cluster. All nodes in the cluster must be reachable to this address. The default is 0.0.0.0.
-client	Specify which client address consul is bound to. This address provides HTTP, DNS, RPC and other services. The default is 127.0.0.1.
-node	The name of the node in the cluster must be unique in a cluster. The default is the host name of the node.
-datacenter	Specify the data center name, the default is dc1.

After starting consul, it will listen to 5 ports by default:
8300: port for replication and leader farwarding
8301: port for lan cossip
8302: port for wan gossip
8500: port for web ui interface
8600: port for viewing node information using dns protocol

The web server starts multiple nginx containers and uses registrator to automatically discover them.

Start multiple nginx containers

Install and use registrator to automatically discover

docker run -d \
--name=registrator \
--net=host \
-v /var/run/docker.sock:/tmp/docker.sock \
--restart=always \
gliderlabs/registrator:latest \
--ip=192.168.116.60 \
consul://192.168.116.70:8500

Viewed on the front end, it has been discovered

Use nginx as a reverse proxy and use Consul-template configuration to automatically modify the configuration file

First download and start the nginx service

#配置nginx官方源，下载并开启
vim /etc/yum.repos.d/nginx.repo
[nginx-stable]
name=nginx stable repo
baseurl=http://nginx.org/packages/centos/7/$basearch/
gpgcheck=0
enabled=1
 
yum install nginx -y
systemctl enable --now nginx

Download consul-template

Layer 4 proxy configuration

There are two configuration files for nginx installed by yum. /etc/nginx/nginx.conf contains the global configuration, events block and http block configuration, and /etc/nginx/conf.d/default.conf contains the server block configuration.

Because the four-layer proxy is done in the global configuration, it is referenced in the nginx global configuration.

Then write a template file to generate and automatically modify the nginx configuration file referenced in the previous step.

#模板文件以ctmpl结尾！
vim nginx.ctmpl
stream {
    upstream nginx_backend {
        #获取consul的nginx服务范围
        {
   
   {range service "nginx"}}
        server {
   
   {.Address}}:{
   
   {.Port}};
        {
   
   {end}}
    }
 
    server {
        listen 9090;
        proxy_pass nginx_backend;
    }
}

Use template files to enable template (foreground startup, background startup plus &)

consul-template --consul-addr 192.168.116.70:8500 \
--template "/opt/consul/nginx.ctmpl:/etc/nginx/template/stream.conf:/usr/sbin/nginx -s reload" \
--log-level=info

Open another terminal and log in to check. The configuration file is automatically modified successfully.

Add a page, access the test, and see if it polls

Layer 7 proxy configuration

Write template file

vim nginx2.ctmpl
upstream nginx_backend {
    #获取consul的nginx服务范围
    {
   
   {range service "nginx"}}
    server {
   
   {.Address}}:{
   
   {.Port}};
    {
   
   {end}}
}
 
server {
    listen 9090;
    location / {
        root /usr/share/nginx/html;
        index index.html;
        proxy_pass http://nginx_backend;
        proxy_set_header HOST $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header Client-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
 
    }
}

You also need to comment the location / {} configuration in /etc/nginx/conf.d/default.conf, otherwise it will conflict with the newly generated location / {} configuration.

conflict.

Start the service and access the test

The configuration of the seven-layer proxy is in the http module, so the configuration file generated here should be under /etc/nginx/conf.d/

Add docker-consul node

Like the node, download the consul service first.

Then enable the consul node and join the first cluster

consul agent \
-server \
-ui \
-data-dir=/var/lib/consul-data \
-bind=192.168.116.60 \
-client=0.0.0.0 \
-node=consul-server02 \
-enable-script-checks=true  \
-datacenter=dc1  \
-join 192.168.116.70 &> /var/log/consul.log &

Docker consul container service automatic discovery and update

1. What is service registration and discovery?