[] Scenarios outlined Zookeeper

• First, the data publish and subscribe (distribution center)
• Second, load balancing
• Third, the Naming Service (Naming Service)
• Fourth, distributed notification / coordination
• V. Cluster Management and Master election
• Sixth, distributed lock
• Seven, distributed transactions

First, the data publish and subscribe (distribution center)

• Publish and subscribe model, the so-called distribution center, by definition it is the publisher will publish data on the node ZK for subscribers dynamic data acquisition, centralized management and dynamic update configuration information. Such as global configuration information, service address lists and other service-service framework is very suitable for use.

Second, load balancing

• Here that means soft load balancing load balancing. In a distributed environment, in order to ensure high availability, usually with the same application or a service provider will deploy more than to reach for other services. And consumers need to select a relevant business logic to perform these peer servers, wherein the message is a typical middleware producers, consumers, load balancing.
• Posted in messaging middleware load balancing and subscribers of, linkedin KafkaMQ open source and open source metaq Ali to do all producers, consumers load balancing by zookeeper. Here, for example, speak to metaq follows:
• Manufacturer Load balancing: When metaq sending the message producer must select a partition on the broker at the time of sending a message to send a message, so metaq during operation, and the broker will all partition information corresponding to all the registered ZK on the specified node, a default strategy is sequentially polling process, after acquiring the list of partitions producer through ZK, in the order in arrangement of partition brokerId and organized into an ordered list of partitions, the transmission time from beginning to end in accordance with iterative manner to send the message to select a partition.
• Consumer Load Balancing: in the consumption process, a consumer will be spending one or more partitions in the news, but a partition will be consumed by a consumer. MetaQ consumer strategy is:
  1. Each partition for the same group only mount a consumer.
  2. If the number of consumers with a group larger than the number of partitions, the more out of the consumer will not participate in consumption.
  3. If the number of consumers with a group of less than the number of partitions, there are some consumers need to undertake additional tasks consumption.
• In the case of a failure or reboot the consumer, other consumers will perceive this change (by zookeeper watch list of consumers), and then re-load balancing to ensure that all partitions have consumers spending.

Third, the Naming Service (Naming Service)

• Naming Service is a distributed system relatively common type of scene. In distributed systems, by using a naming service, the client application able to get the address of the specified resource or service name, informants and so on. Named entity can usually cluster of machines, service address provided, the remote object, and so on - which we can refer to them as the name (Name). One of the more common is some distributed service framework service address lists. By calling the API to create a node ZK provided, can easily create a globally unique path, this path can be used as a name.
• Alibaba Group using open source framework for distributed services in Dubbo ZooKeeper as its naming service, maintain a global address list of services, click here to see Dubbo open source project. In Dubbo implementation:
• Service provider when it starts, write your URL address to the designated node on ZK / dubbo / $ {serviceName} under / providers directory, this operation is complete publishing services.
• Consumer service startup, subscribe / dubbo / \ ({serviceName} provider under / providers directory URL address, and / dubbo / \) write their own URL address / consumers under the directory {serviceName}.
• Note that all registered with the ZK node addresses are temporary, so we can guarantee service providers and consumers can automatically sense changes in resources. In addition, Dubbo there are services for monitoring granularity, by subscription / dubbo / $ {serviceName} directory of all providers and consumers of information.

Fourth, distributed notification / coordination

　　ZooKeeper peculiar watcher registration and asynchronous notification mechanism to achieve a good notification and coordination among the different systems in a distributed environment, real-time processing of the data changes. Usually use different systems are all on the same znode ZK register, monitor the changes znode (including znode own content and child nodes), one of the znode system update, then another system to receive notifications and respond accordingly deal with

1. Another heartbeat detection: between the detection system and the detection system are not directly associated, but by a node associated with the ZK, greatly reducing the coupling system.
2. Another system scheduling modes: a system console, and a push system composed of two parts, the control console functions corresponding push push system work. In some operations manager for the console, in fact, is to modify the status of certain nodes on ZK, ZK put these changes and notify them of the registration Watcher client, namely push system, then make the appropriate push task.
3. Another report on the work mode: something similar task distribution system, the subtasks started to zk to register a temporary node, and the timing of their progress to report (to write back the progress of this temporary node), so the task manager on You can know in real time the progress of the task.
   In summary, using zookeeper notification and coordination for distributed can greatly reduce the coupling between the system

V. Cluster Management and Master election

1. Cluster Machine Monitoring: This is usually the kind used to cluster state of the machine, the machine has a scene online rate higher requirements, can quickly respond to changes in the cluster machine. Such a scenario, tend to have a monitoring system, whether real-time detection machine cluster survival. Past practice is usually: monitoring system by some means (such as ping) to detect the timing of each machine or each machine its own regular reports to the monitoring system, "I'm alive." This approach works, but there are two obvious problems:
2. Cluster machines are subject to change when more implicated modify things.
3. A certain delay. Use ZooKeeper has two characteristics, another cluster machine liveness monitoring system can be achieved:
4. On the client node x registered a Watcher, then if x? Child of change, and will notify the client.
5. Creating EPHEMERAL types of nodes, once the session client and server end or expired, then the node will disappear.
   • For example, the monitoring system on / clusterServers a Watcher node registration, dynamic added after every machine, create a EPHEMERAL aboard the node type / clusterServers: / clusterServers / {hostname} Thus, the monitoring system will be able to know in real time by the machine. Save the case, as for the subsequent processing operations that the monitoring system.
6. Master election is a zookeeper in the most classic scenarios.
   • In a distributed environment, the same business applications distributed on different machines, some business logic (e.g., a number of time-consuming calculation, network I / O process), often only one of the machines to make the entire cluster for execution, the remaining machines can share the results, which can greatly reduce duplication and improve performance, so the master election is the main problem encountered in this scenario.
   • ZooKeeper use strong consistency, to ensure the nodes created under high concurrency distributed global uniqueness, namely: there are multiple client requests to create / currentMaster node, ultimately only a certain client requests to create success. Using this feature, it can easily be selected clusters in a distributed environment.
   • In addition, look at the evolution of this scenario, it is the dynamic Master election. This use to EPHEMERAL_SEQUENTIAL characteristics of the type of node.
   • All client requests to create, ultimately only one able to create success. In this slight variation, is to allow all requests can be created, but have to have the order of creation, then all requests to create the final result on ZK One possible situation is this: / currentMaster / {sessionId} -1, / currentMaster / {sessionId} -2, / currentMaster / {sessionId} -3 ... .. each machine to select the smallest serial number as the Master, if linked to the machine, since the node will immediately create his hours, then after a minimum that is the Master of the machine.
7. In search system, if each machine in the cluster to generate a full index amount, time consuming and does not guarantee consistency between the index data with each other. So let the cluster Master to generate full amount of the index, and then sync to other machines in the cluster. In addition, disaster recovery measures Master election that can be manually specify the master at any time, that when used in zk can not obtain master information, you can obtain master to a place such as by way of http.
8. In Hbase, the election is to achieve dynamic HMaster use ZooKeeper. In Hbase implementation, ZK will be stored on the table and some addresses ROOT HMaster address, registered to the Zookeeper HRegionServer will put themselves in a temporary node (Ephemeral) a way that HMaster can perceive survival status of each HRegionServer at any time, At the same time, once HMaster problems, will be re-elected to a HMaster run, thus avoiding the problem of single point HMaster

Sixth, distributed lock

　　Distributed Lock, this is mainly due to the strong ZooKeeper for us to ensure data consistency. Lock service can be divided into two categories, one is to maintain exclusivity, the other is to control the timing.

1. Keep the so-called exclusive, is that all attempts to acquire the lock of the client, and ultimately only one can successfully get the lock. The usual practice is to put on a znode zk seen as a lock, which is achieved by create znode way. All clients to create / distribute_lock nodes, that create the ultimate success of the client that is owned the lock.
2. Control of the timing is that all views to get the client the lock, and ultimately will be scheduled for execution, but there is a global timing up. And substantially similar to the above practices, where only / distributelock already pre-existing, the client creates an ordered temporary node (which can be controlled by property node: CreateMode.EPHEMERALSEQUENTIAL specified) below it. Zk parent node (/ distribute_lock) maintain a sequence, to ensure that the timing of the creation of a child node, thus forming a global timing for each client.

Seven, distributed transactions