Consul overview

What is service registration and discovery
Service registration and discovery are indispensable and important components in the microservice architecture. At first, the services were single-node, which did not guarantee high availability and did not consider the pressure bearing of the service. The calls between services were simply accessed through the interface. Until the distributed architecture of multiple nodes appeared later, the initial solution was to balance the load on the service front-end, so that the front-end must know the network locations of all back-end services and configure them in the configuration file. There will be a few questions here:

1) If you need to call the backend service AN, you need to configure the network location of N services, which is very troublesome to configure.
2) If the network location of the backend service changes, you need to change the configuration of each caller

Since there are these problems, service registration and discovery are to solve these problems. The back-end service AN can register its current network location to the service discovery module, and the service discovery will be recorded in the form of KV. K is generally the service name, and V is IP:PORT. The service discovery module conducts health checks regularly, and polls to see if these back-end services can be accessed. When the front-end calls the back-end service AN, it runs to the service discovery module to ask their network locations, and then calls their services. In this way, the above problems can be solved. The front-end does not need to record the network locations of these back-end services at all, and the front-end and back-end are completely decoupled.

Installing consul is used for service registration, that is, some information of the container itself is registered in consul, and other programs can obtain registered service information through consul, which is service registration and discovery

The concept of consul

consul is a service management software developed by google open source using go language. Supports multi-data centers, distributed high availability, service discovery and configuration sharing. The Raft algorithm is used to ensure high availability of services. Built-in service registration and discovery framework, distributed consistency protocol implementation, health check, Key/Value storage, multi-data center solution, no longer need to rely on other tools (such as ZooKeeper, etc.). Service deployment is simple, with only one executable binary package. Each node needs to run the agent, which has two operating modes server and client. each number

According to the official recommendation of the center, 3 or 5 server nodes are required to ensure data security, and at the same time ensure that the election of server-leader can be carried out correctly

Consul mode:

1) In the client mode, all services registered to the current node will be forwarded to the server node, which does not persist the information itself 2)
In the server mode, the function is similar to the client mode, the only difference is that it will send all
The information is persisted to the local, so that in case of a failure, the information can be retained Nodes are also responsible for the health monitoring of each node

The role of Consul

Service registration and discovery (main functions), providing two discovery methods of HTTP and DNS
Health check, supporting multiple protocols, HTTP, TCP and other
Key/Value storage
Supporting multiple data centers
Based on Golong language, strong portability
Supporting ACL access control
and Lightweight containers like Docker work seamlessly

Some key features provided by consul

Service registration and discovery: Consul makes service registration and service discovery easy through the DNS or HTTP interface. Some external services, such as those provided by saas, can also register health checks: health checks enable consul to quickly alert operations
in the cluster . Integration with service discovery prevents services from forwarding to failed services
Key/Value storage: a system for storing dynamic configurations. Provides a simple HTTP interface that can be operated anywhere
Multi-data center: supports any number of regions without complicated configuration

consul-template overview

Consul-Template is a daemon process, which is used to query Consul cluster information in real time.
Consul-Template can update any number of specified templates on the file system and generate configuration files.
1) After the update is completed, you can choose to run the shell command to perform the update operation and reload Nginx
Consul-Template can query the service directory, Key, Key-values, etc. in Consul.
This powerful abstraction and query language templates make Consul-Template especially suitable for dynamically creating configuration files.
1) For example: Create Apache/Nginx Proxy Balancers, Haproxy Backends

The role of the registrator

A service written in Go language for docker that can be used to detect container status, automatically register and unregister docker containers to the service configuration center. Consul, Etcd, and SkyDNS2 are currently supported.

Build a consul cluster environment

1. Install Consul

1) Compile and install consul

mkdir /root/consul
cd /root/consul
//放入安装包
unzip consul_0.9.2_linux_amd64.zip
mv consul /usr/bin

consul agent \
-server \
-bootstrap \
-ui \
-data-dir=/var/lib/consul-data \
-bind=192.168.163.143 \
-client=0.0.0.0 \
-node=consul-server01 &> /var/log/consul.log &

2) View cluster information

consul members

consul info | grep leader
这里查询到的8300端口用于集群内数据的读写和复制

3) Obtain cluster information through httpd api

curl 127.0.0.1:8500/v1/status/peers       //查看集群server成员
curl 127.0.0.1:8500/v1/status/leader      //集群Raf leader
curl 127.0.0.1:8500/v1/catalog/services   //注册的所有服务
curl 127.0.0.1:8500/v1/catalog/nginx      //查看nginx服务信息
curl 127.0.0.1:8500/v1/catalog/nodes      //集群节点详细信息

netstat -natp |grep consul
这5个端口的作用：
8300：集群内数据的读写和复制
8301：单个数据中心gossip协议通讯
8302：跨数据中心gossip协议通讯
8500：提供获取服务列表、注册服务、注销服务等HTTP接口；提供UI服务
8600：采用DNS协议提供服务发现功能

2. Configure Container Service to automatically join the nginx cluster

1) Install Gliderlabs/Registrator

docker run -d \
--name=registrator \
--net=host \
-v /var/run/docker.sock:/tmp/docker.sock \
--restart=always \
gliderlabs/registrator:latest \
-ip=192.168.163.142 \
consul://192.168.163.143:8500

2) Test service
to test whether the function is normal

docker run -itd -p:81:80 --name test-01 -h test01 nginx
docker run -itd -p:82:80 --name test-02 -h test02 nginx
docker run -itd -p:91:80 --name test-03 -h test03 httpd
docker run -itd -p:92:80 --name test-04 -h test04 httpd

Prepare template nginx template file

vim /root/consul/nginx.ctmpl

upstream http_backend {
  {
   
   {range service "nginx"}}
  server {
   
   {.Address}}:{
   
   {.Port}};
  {
   
   {end}}
}

server {
  listen 81;
  server_name localhost 192.168.163.143;
  access_log /var/log/nginx/nginx01-access.log;
  index index.html index.php;
  location / {
  proxy_set_header HOST $host;
  proxy_set_header X-Real-IP $remote_addr;
  proxy_set_header Client-IP $remote_addr;
  proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  proxy_pass http://http_backend;
  }
}

Deploy consul multi-node

Add a server with an existing docker environment

mkdir /root/consul
cd /root/consul
//放入安装包
unzip consul_0.9.2_linux_amd64.zip
mv consul /usr/bin

consul agent \
-server \
-bootstrap \
-ui \
-data-dir=/var/lib/consul-data \
-bind=192.168.163.139 \
-client=0.0.0.0 \
-node=consul-server02 \
-enable-script-checks=true \
-datacenter=dc1 \
-join 192.168.163.143 &> /var/log/consul.log &

#--------参数解释--------------------------------
-enable-script-checks=true:设置检查服务为可用
-datacenter:数据中心名称
-join:加入到已有的集群中

Docker consul cluster