The more we have the service, if we use a predefined port, the greater the potential for conflict. After all, on the same port can not monitor the two services. Close list itself all the ports of one hundred management service used is a challenge. Adding those databases required services to the list, the number will grow even more. For this reason, we should deploy services without specifying the port, and let Docker for us to assign a random service. The only problem is that we need to find the port number and let others know about it.
When we started to deploy multiple servers to one service a distributed system, things become more complicated. We can choose which service to which pre-defined server but can cause a lot of problems. We should make use of server resources as much as possible, if we define in advance the deployment of each service where it is almost impossible. Another problem is automatically extended services are the most difficult, not to mention the automatic recovery of a server failure. On the other hand, if we will deploy the service to the server (for example, running a minimum number of containers), we need to add the IP to the list of data need to find somewhere and stored.
When we need to store and retrieve (find) some of the information related to the services we are using, there are many other cases.
In order to be able to find our services, we need at least the following two processes for our use.
- Service registration process, at least the host and port services stores that are running.
- Service discovery process, allowing others to find the information we store in the registration process.
In addition to these processes, we also need to consider several other aspects. If the service stopped working and deploy / register a new instance, whether we should cancel the registration of the service? What happens when there are multiple copies of the same service? How do we balance the load between them? If a server fails what will happen? These and many other issues related to the registration and discovery process are closely related. Currently, we only restricts the scope to service discovery (including generic names of the two processes) as well as the tools we may for such tasks. Most of them have some kind of high-availability distributed key / value store.
Service Discovery Tool
Service discovery tool main goal is to help the service to find and communicate with each other. In order to perform their duties, they need to know the location of each service. This concept is not new, many tools Docker was born long before existed. However, the container needs such a tool has reached a whole new level.
Service discovery The basic idea behind the service (or application) for each new instance can identify their current environment and store the information. Itself is typically stored in the registry to perform the key / value format. Since this discovery is generally used in a distributed system, the registry is scalable, distributed among all the nodes in the cluster and fault-tolerant. The main purpose of this storage is at least may need to provide services to communicate with all parties of IP and port. These data are usually extended with other types of information.
Discovery tools tend to provide some kind of API, the service can use it to register themselves and others to find information about the service.
Suppose we have two services. One is the provider, and the other is the consumer. After the deployment provider, we need to store information selected the service discovery registry in. Later, when the consumer attempts to access provider, it will first query the registry and use the IP and port of call providers obtained from the registry. In order to provide a specific person with the registry of separation, we often use some kind of proxy service . In this way, consumers will always request information from a fixed address that resides within the agency, and the agency in turn use the discovery service provider to find out information and redirect the request. We will reverse proxy later in the book. Now, to understand the process based on three participants is very important; consumers, agents and providers.
We found the service tool to find the data. At least we should be able to find the location of services, and health services are configured services. Since we are building a distributed system with multiple servers, so the need for robust tools, and a node failure should not endanger data. In addition, each node should have the exact same copy of the data. In addition, we hope to be able to start the service in any order, capable of destroying them or replace them with a newer version. We should also be able to reconfigure our services and accordingly to view the data.
Let's look at several options to accomplish common goals we have set.
Manual configuration
Most services are still manual management. We decide in advance where to deploy the service, what its configuration is, and hope that it will continue to work until the end. This method is difficult to expand. The second instance of the deployment of services means that we need to fully activate the manual process. We need to start a new server or to find out which resource utilization is low, create a new set of configuration and deployment. In the case of hardware failure situation is more complicated, because when manually managing things, the reaction time is usually very slow. Visibility is another painful point of view. We know what static configuration Yes. After all, we are ready in advance. However, most services have a large number of dynamically generated information. This information is not easy to see. When we need these data, we can not query a single location.
Since a large number of moving parts of the manual operation, the reaction time inevitably slow, resiliency most difficult to manage and to monitor suspicious.
Although the work done manually in the past or services and / or a small number of servers is that there is an excuse, but with the emergence of service discovery tool, that excuse will soon disappear.
zookeeper, referred zk, Zookeeper
ZooKeeper is one of the oldest of this type of project. It originated in the Hadoop world, it is to help maintain the various components of Hadoop cluster constructed. It is mature, reliable, and many large companies (YouTube, eBay, Yahoo, etc.) to use. It stores data format similar to the organization of the file system. If running on a server cluster, Zookeper configuration state shared among all the nodes. Each cluster select a leader, clients can connect to any server to retrieve data.
The main advantage is Zookeeper brought maturity, robustness and feature-rich. However, it also has its own disadvantages, Java and complexity are the main culprit. Although Java for many use cases are great, but for this kind of work, it is very heavy. Zookeeper use of Java as well as a considerable number of dependencies makes it more scarce resources in the competition. In addition to these problems, Zookeeper is also very complicated. It needs to maintain than what we expect from this type of application more knowledge. This is a feature-rich advantages into itself from the portion of liabilities. The more applications with functionality chance we do not need all the features will be. So, we end up is not exactly what we need to pay the price in a complicated form.
Zookeeper paved the way to other people, and made considerable progress. "Big players" are using it, because there was no better choice. Today, Zookeeper shows its age, our choices better.
ETCD
etcd is a bond accessible via HTTP / value storage. It is distributed, hierarchical configuration having a system for building service discovery. It is easy to deploy, set up and use, provide reliable data persistence, security, and very good documentation.
Because of its simplicity, etcd than Zookeeper better choice. However, it needs to provide services in conjunction with a small number of third-party tools to find the target.
现在我们有了一个存储与我们服务相关的信息的地方，我们需要一个能够自动将该信息发送给etcd的工具。毕竟，如果可以自动完成，我们为什么要手动将数据输入到etcd。即使我们想要将信息手动输入到etcd，我们通常也不知道该信息是什么。请记住，服务可能部署到运行最少容器的服务器，并且可能已分配随机端口。理想情况下，该工具应监视所有节点上的Docker，并在运行新容器或停止现有容器时更新etcd。可以帮助我们实现这一目标的工具之一是Registrator。
Registrator
Registrator通过检查容器在线或停止时自动注册和取消注册服务。它目前支持etcd，Consul和SkyDNS 2。
Registrator结合etcd是一个强大而简单的组合，它允许我们练习许多高级技术。每当我们调出容器时，所有数据都将存储在etcd中并传播到集群中的所有节点。我们对这些信息的处理取决于我们。
还有一块拼图丢失了。我们需要一种方法来创建配置文件，其中包含存储在etcd中的数据，以及在创建这些文件时运行一些命令。
confd
confd是一个轻量级的配置管理工具。常见用法是使用存储在etcd，consul和少数其他数据注册表中的数据使配置文件保持最新。它还可用于在配置文件更改时重新加载应用程序。换句话说，我们可以将它用作使用etcd（或许多其他注册表）中存储的信息重新配置所有服务的方法。
关于etcd，Registrator和confd组合的最终想法
当etcd，Registrator和confd相结合时，我们会得到一种简单而强大的方法来自动化我们所有的服务发现和配置需求。这种组合还证明了正确组合“小”工具的有效性。这三个人正是我们需要他们做的事情。不到这一点，我们就无法实现摆在我们面前的目标。另一方面，如果它们的设计考虑到更大的范围，我们会在服务器资源上引入不必要的复杂性和开销。
在我们做出最终判决之前，让我们看一下具有类似目标的另一种工具组合。毕竟，我们不应该在没有调查替代方案的情况下解决某些问题。
Consul
Consul是一个强大的一致数据存储区，它使用八卦来形成动态集群。它具有分层键/值存储，不仅可用于存储数据，还可用于注册可用于各种任务的监视，从发送有关数据更改的通知到运行运行状况检查和自定义命令（取决于其输出）。
与Zookeeper和etcd不同，Consul实现嵌入式服务发现系统，因此无需构建自己的服务或使用第三方服务。此发现包括对其上运行的节点和服务的运行状况检查。
ZooKeeper和etcd仅提供原始的K / V存储，并要求应用程序开发人员构建自己的系统以提供服务发现。另一方面，Consul为服务发现提供了内置框架。客户端只需要使用DNS或HTTP接口注册服务并执行发现。其他两个工具需要手工解决方案或使用第三方工具。
Consul为多个数据中心和八卦系统提供开箱即用的本机支持，这些系统不仅适用于同一集群中的节点，也适用于数据中心。
Consul有另一个很好的功能，可以区别于其他功能。它不仅可用于发现有关已部署服务及其所在节点的信息，还可通过HTTP请求，TTL（生存时间）和自定义命令轻松扩展运行状况检查。
Registrator
Registrator有两个Consul协议。所述consulkv协议产生类似的结果作为与该ETCD协议获得的那些。
除了通常与etcd或consulkv协议一起存储的IP和端口之外，Registrator的consul协议还存储了更多信息。我们获取有关运行服务的节点的信息以及服务ID和名称。除了少量额外的环境变量，我们还可以以标签的形式存储其他信息
consul模板
confd可以与consul一样使用，与etcd一样。然而，Consul拥有自己的模板服务，其功能更符合Consul提供的服务。
该模板是建立与confd获得的值文件非常方便的方式。作为额外的奖励，它还可以在文件更新后运行任意命令。就像confd一样，consul-template也使用Go Template格式。
consul健康检查，Web UI和数据中心
监视群集节点和服务的运行状况与测试和部署本身一样重要。虽然我们应该致力于拥有永不失败的稳定环境，但我们也应该承认意外的失败发生并准备采取相应的行动。例如，我们可以监视内存使用情况，如果达到某个阈值，则将某些服务移动到集群中的其他节点。这将是在“灾难”发生之前执行的预防措施的一个例子。另一方面，并非所有潜在的故障都能及时检测到我们按时采取行动。单个服务可能会失败。由于硬件故障，整个节点可能会停止工作。在这种情况下，我们应该准备尽可能快地采取行动，例如，用新的节点替换节点并移动失败的服务。领事有一个简单，优雅，
如果您搜索“etcd ui”或“etcd仪表板”，您可能会看到有一些可用的解决方案，可能会问我们为什么没有提供它们。原因很简单; etcd是一个键/值存储，而不是更多。拥有用于呈现数据的UI并没有多大用处，因为我们可以通过etcdctl轻松获取它。这并不意味着etcd UI没有用，但由于其范围有限，它没有太大的区别。
Consul不仅仅是一个简单的键/值存储。正如我们已经看到的，除了存储简单的键/值对之外，它还具有服务概念以及属于它的数据。它还可以执行运行状况检查，从而成为仪表板的良好候选者，可用于查看在其上运行的节点和服务的状态。最后，它理解多个数据中心的概念。所有这些功能相结合，让我们看到了对不同灯光仪表板的需求。
使用Consul Web UI，我们可以查看所有服务和节点，监控运行状况检查及其状态，读取和设置键/值数据以及从一个数据中心切换到另一个数据中心。
关于Consul，Registrator，Template，健康检查和Web UI的最终想法
在许多情况下，Consul与我们探索的工具一起比etcd提供的解决方案更好。它的设计考虑了服务架构和发现。它很简单，但功能强大。它提供了一个完整的解决方案，而且不会牺牲简单性，在许多情况下，它是服务发现和健康检查需求的最佳工具。
结论
所有工具都基于类似的原则和架构。它们在节点上运行，需要仲裁才能运行并且非常一致。它们都提供某种形式的键/值存储。
zk是三个中最老的，年龄表现在其复杂性，资源利用和目标。它的设计时间与我们评估的其他工具的年龄不同（即使它不是很老）。
ETCD与Registrator和confd是可以解决大部分，如果不是所有的我们的服务发现的需求，一个非常简单但非常强大的组合。它还展示了我们在结合简单和非常具体的工具时可以获得的强大功能。它们中的每一个都执行一项非常具体的任务，通过完善的API进行通信，并且能够使用相对自治。他们自己在建筑和功能方面都是微服务。
Consul的不同之处在于，无需使用第三方工具即可支持多个数据中心和健康检查。这并不意味着使用第三方工具很糟糕。实际上，在整个博客中，我们尝试通过选择性能优于其他工具而不引入不必要的功能开销来组合不同的工具。当我们使用正确的工具完成工作时，可以获得最佳结果。如果该工具的功能超过我们所需的工作，其效率就会下降。另一方面，不做我们需要它的工具是没用的。领事达到了正确的平衡。它做的事情很少，而且做得很好。
Consul use gossip spread knowledge clusters way that is easier than setting etcd, especially in the case of large data centers. The data is stored as the ability to service makes it key / value store etcd in a more complete and useful (although Consul also has this option) ratio. While we can insert the plurality of keys etcd to accomplish the same, but the service may be implemented Consul results more compact, generally requires a single query to retrieve all the data related to the service. The most important thing is, Registrator has a very good consular protocol, making it a good combination of both, especially when the consul-template is added to this picture when. Consul's Web UI is like the cherry on a cake, a good way to provide a visual service and health.
I can not say Consul is a clear winner. On the contrary, compared with etcd, it is a slight advantage. As a concept and a service discovery tool we can use is so new that we can expect a lot of changes in this area. To have an open mind, and try to learn some advice from this article. Try different tools and make their own conclusions.
Translated from: https://technologyconversations.com/2015/09/08/service-discovery-zookeeper-vs-etcd-vs-consul/