Docker's Consul-based service discovery Cluster Setup

In last year's .NET Core series micro service , the initial learning a bit Consul service discovery, summed up the two articles . The deployment of Docker-based approach to a Demo example to build an example of a cluster of Consul, and finally gives a demonstration HA architecture, it will be closer to the actual application environment.

First, an example of the overall architecture

　　This example will be made a API Gateway, a three Consul Consul Client and Server composition, background knowledge of Client and Server Agent related to these two modes of Consul, please move them to understand my previous article: " the .NET Micro Core Services the Consul achieve service-based governance . " Which, Consul of Client and Server nodes together constitute a Data Center, while the API Gateway acquired from Consul to IP and port number of the service, and returned to the service consumer. Here's API Gateway is that it is not focused on Ocelot implemented here, but there is more than explained, my friends do not understand the venue of another article: " the .NET Micro Core service based on the Ocelot implement the API Gateway Service . "

Two, Consul Cluster Setup

2.1 Consul mirrored pull

docker pull consul:1.4.4

　　Verify: docker images

　　

2.2 Consul Server instance creation

　　The following is my practice (CentOS 7) operate on a single machine, thus using the three examples are different port numbers (different from the default port number 8500). The actual environment, it is recommended to deploy multiple machines.

　　(1) Example 1 Consul

docker run -d -p 8510:8500 --restart=always -v /XiLife/consul/data/server1:/consul/data -v /XiLife/consul/conf/server1:/consul/config -e CONSUL_BIND_INTERFACE='eth0' --privileged=true --name=consul_server_1 consul:1.4.4 agent -server -bootstrap-expect=3 -ui -node=consul_server_1 -client='0.0.0.0' -data-dir /consul/data -config-dir /consul/config -datacenter=xdp_dc;

　　(2) Example 2 Consul

　　In order to join the cluster Consul Example 2, first of all get what IP address Consul Example 1:

　　JOIN_IP="$(docker inspect -f '{{.NetworkSettings.IPAddress}}' consul_server_1)";

docker run -d -p 8520:8500 --restart=always -v /XiLife/consul/data/server2:/consul/data -v /XiLife/consul/conf/server2:/consul/config -e CONSUL_BIND_INTERFACE='eth0' --privileged=true --name=consul_server_2 consul:1.4.4 agent -server -ui -node=consul_server_2 -client='0.0.0.0' -datacenter=xdp_dc -data-dir /consul/data -config-dir /consul/config -join=$JOIN_IP;　　

　　(3) Example 3 Consul

docker run -d -p 8530:8500 --restart=always -v /XiLife/consul/data/server3:/consul/data -v /XiLife/consul/conf/server3:/consul/config -e CONSUL_BIND_INTERFACE='eth0' --privileged=true --name=consul_server_3 consul:1.4.4 agent -server -ui -node=consul_server_3 -client='0.0.0.0' -datacenter=xdp_dc -data-dir /consul/data -config-dir /consul/config -join=$JOIN_IP;

　　验证1：docker exec consul_server_1 consul operator raft list-peers

　　

　　验证2：http://192.168.16.170:8500/

　　

2.3 Consul Client实例创建

　　（1）准备services.json配置文件，向Consul注册两个同样的Product API服务

    {
      "services": [
        {
          "id": "core.product-/192.168.16.170:8000",
          "name": "core.product",
          "tags": [ "xdp-/core.product" ],
          "address": "192.168.16.170",
          "port": 8000,
          "checks": [
            {
              "name": "core.product.check",
              "http": "http://192.168.16.170:8000/api/health",
              "interval": "10s",
              "timeout": "5s"
            }
          ]
        },
        {
          "id": "core.product-/192.168.16.170:8001",
          "name": "core.product",
          "tags": [ "xdp-/core.product" ],
          "address": "192.168.16.170",
          "port": 8001,
          "checks": [
            {
              "name": "core.product.check",
              "http": "http://192.168.16.170:8001/api/health",
              "interval": "10s",
              "timeout": "5s"
            }
          ]
        }
      ]
    }

　　有关配置文件的细节，请移步另一篇文章：《.NET Core微服务之基于Consul实现服务治理（续）》

　　（2）Consul Client实例

docker run -d -p 8550:8500 --restart=always -v /XiLife/consul/conf/client1:/consul/config -e CONSUL_BIND_INTERFACE='eth0' --name=consul_client_1 consul:1.4.4 agent -node=consul_client_1 -join=$JOIN_IP -client='0.0.0.0' -datacenter=xdp_dc -config-dir /consul/config

　　（3）验证

　　

　　

　　

2.4 服务检查监控邮件提箱

　　（1）为Client添加watches.json

       {
          "watches": [
            {
              "type": "checks",
              "handler_type": "http",
              "state": "critical",
              "http_handler_config": {
                "path": "http://192.168.16.170:6030/api/Notifications/consul",
                "method": "POST",
                "timeout": "10s",
                "header": { "Authorization": [ "token" ] }
              }
            }
          ]
        }

　　*.这里的api接口 http://192.168.16.170:6030/api/Notifications/consul是我的一个通知服务接口，下面是实现的代码

        /// <summary>
        /// 发送Consul服务中心健康检查Email
        /// </summary>
        [HttpPost("consul")]
        public async Task SendConsulHealthCheckEmail()
        {
            using (var stream = new MemoryStream())
            {
                HttpContext.Request.Body.CopyTo(stream);
                var ary = stream.ToArray();
                var str = Encoding.UTF8.GetString(ary);

                dynamic notifications = JsonConvert.DeserializeObject(str);
                if (notifications == null || notifications.Count == 0)
                {
                    return;
                }

                var title = "XDP服务中心健康检查通知";
                var emailBody = new StringBuilder($"<span style='font-weight:bold; color:red;'>{title}</span> : <br/>");
                foreach (var notification in notifications)
                {
                    emailBody.AppendLine($"---------------------------------------------------------<br/>");
                    emailBody.AppendLine($"<span style='font-weight:bold;'>节点</span>:{notification.Node}<br/>");
                    emailBody.AppendLine($"<span style='font-weight:bold;'>服务ID</span>:{notification.ServiceID}<br/>");
                    emailBody.AppendLine($"<span style='font-weight:bold;'>服务名称</span>:{notification.ServiceName}<br/>");
                    emailBody.AppendLine($"<span style='font-weight:bold;'>检查ID</span>:{notification.CheckID}<br/>");
                    emailBody.AppendLine($"<span style='font-weight:bold;'>检查名称</span>:{notification.Name}<br/>");
                    emailBody.AppendLine($"<span style='font-weight:bold;'>检查状态</span>:{notification.Status}<br/>");
                    emailBody.AppendLine($"<span style='font-weight:bold;'>检查输出</span>:{notification.Output}<br/>");
                    emailBody.AppendLine($"---------------------------------------------------------<br/>");
                }

                var email = new Email()
                {
                    Username = _configuration["EmailSettings:Username"],
                    Password = _configuration["EmailSettings:Password"],
                    SmtpServerAddress = _configuration["EmailSettings:SmtpServerAddress"],
                    SmtpPort = Convert.ToInt32(_configuration["EmailSettings:SmtpPort"]),
                    Subject = title,
                    Body = emailBody.ToString(),
                    Recipients = _configuration["EmailSettings:Recipients"]
                };

                email.Send();
            }
        }

            /// <summary>
            /// 使用同步发送邮件
            /// </summary>
            public void Send()
            {
                using (SmtpClient smtpClient = GetSmtpClient)
                {
                    using (MailMessage mailMessage = GetClient)
                    {
                        if (smtpClient == null || mailMessage == null) return;
                        Subject = Subject;
                        Body = Body;
                        //EnableSsl = false;
                        smtpClient.Send(mailMessage); //异步发送邮件,如果回调方法中参数不为"true"则表示发送失败
                    }
                }
            }    

View Code

　　（2）验证

　　

三、Ocelot网关配置

3.1 为Ocelot增加Consul支持

　　（1）增加Nuget包：Ocelot.Provider.Consul

Nuget>> Install-Package Ocelot.Provider.Consul　　

　　（2）修改StartUp.cs，增加Consul支持

s.AddOcelot()
    .AddConsul();

　　更多内容，请移步：Ocelot官方文档-服务发现

3.2 修改Ocelot配置文件增加Consul配置

    "GlobalConfiguration": {
        "BaseUrl": "http://api.xique.com",
        "ServiceDiscoveryProvider": {
             "Host": "192.168.16.170",
             "Port": 8550,
             "Type": "Consul"
        }
    }

　　*.这里指向的是Consul Client实例的地址

　　此外，Ocelot默认策略是每次请求都去Consul中获取服务地址列表，如果想要提高性能，也可以使用PollConsul的策略，即Ocelot自己维护一份列表，然后定期从Consul中获取刷新，就不再是每次请求都去Consul中拿一趟了。例如下面的配置，它告诉Ocelot每2秒钟去Consul中拿一次。

    "Type": "PollConsul",
    "PollingInterval": 2000

3.3 Service配置

    // -- Service
    {
      "UseServiceDiscovery": true,
      "DownstreamPathTemplate": "/api/{url}",
      "DownstreamScheme": "http",
      "ServiceName": "core.product",
      "LoadBalancerOptions": {
        "Type": "RoundRobin"
      },
      "UpstreamPathTemplate": "/product/{url}",
      "UpstreamHttpMethod": [ "Get", "Post", "Put", "Delete" ]
    }

　　这里配置了在Consul中配置的服务名（ServiceName），以及告诉Ocelot我们使用轮询策略（RoundRobin）做负载均衡。

3.4 验证

　　第一次访问：

　　

　　第二次访问：

　　

四、HA示例整体架构

　　对于实际应用中，我们往往会考虑单点问题，因此会借助一些负载均衡技术来做高可用的架构，这里给出一个建议的HA示例的整体架构：

　　对于一个API请求，首先会经历一个Load Balancer才会到达API Gateway，这个Load Balancer可以是基于硬件的F5，也可以是基于软件的Nginx或LVS再搭配Keepalived，一般来说大部分团队都会选择Nginx。然后API Gateway通过部署多个，来解决单点问题，也达到负载均衡的效果。而对于API Gateway和Consul Client之间的连接，我们往往也会增加一个Load Balancer来实现服务发现的高可用，这个Load Balancer也一般会基于Nginx/LVS搭配Keepalived，API Gateway只需要访问一个Virtual IP即可。而在Consul Data Center中，Consul Server会选择3或5个，Consul Client也会部署多个，刚刚提到的Virtual IP则会指向多个Consul Client，从而防止了Consul Client的单点问题。

　　最后，祝大家端午安康！

　　

　　

作者：周旭龙

出处：http://edisonchou.cnblogs.com

本文版权归作者和博客园共有，欢迎转载，但未经作者同意必须保留此段声明，且在文章页面明显位置给出原文链接。