ZooKeeper
1. Overview
Zookeeper is a tool that enables distributed coordination services in a cluster.
The so-called distributed coordination service is to perform reliable message delivery among the nodes of the cluster to coordinate the work of the cluster.
The reason why Zookeeper can implement distributed coordination services is that it can ensure distributed data consistency.
The so-called distributed data consistency means that the consistency of data transmission can be guaranteed in the cluster.
The distributed coordination services that Zookeeper can provide include: data publishing and subscription, load balancing, naming services, distributed coordination/notification, cluster management, distributed locks, distributed queues, etc.
1. Features of Zookeeper
Zookeeper works in the cluster and provides distributed coordination services for the cluster. The distributed coordination services it provides have the following characteristics:
1. sequential consistency
Transaction requests initiated from the same client will eventually be applied to zookeeper in strict accordance with the order in which they were initiated
2. atomicity
The application of the processing results of all transaction requests on all machines in the entire cluster is consistent, that is, either all machines in the entire cluster have successfully applied a certain transaction, or none of them have been applied, and there will be no application of some machines in the cluster. In order to change the transaction, the other part is not applied.
3. single view
No matter which zookeeper server the client connects to, the server-side data model it sees is the same.
4. reliability
Once the server successfully applies a transaction and completes the response to the client, the server state changes caused by the transaction will remain until another transaction changes it.
5. real-time
Zookeeper is not a strong consistency, it can only guarantee sequential consistency and final consistency, which can only be called pseudo real-time.
2. Data Model
Data can be saved in zookeeper. It is precisely by using the feature that zookeeper can save data that our cluster transmits messages by accessing data in zookeeper.
1. Data structure
The structure of saving data in zookeeper is very similar to the file system. They are all tree structures composed of nodes. The difference is that the file system is a tree composed of folders and files, while zookeeper is a tree composed of ZNODEs.
Each ZNODE can store a piece of data, and zero or more sub-ZNODE nodes can be mounted under the ZNODE to form a tree structure.
2. Data model classification
Nodes include sequential nodes, ordinary nodes, temporary nodes, and persistent nodes. The four intersections produce the following four actual node types.
| sequential node | normal node | |
| temporary node | sequential ephemeral nodes | ordinary temporary node |
| persistent node | Number ordinal persistent node | normal persistent node |
Sequence node: specify what it is called, except that the prefix is the specified name, there will be a unique and automatically increasing number after the name.
Ordinary node: Call it whatever you specify.
Ephemeral Nodes: Ephemeral nodes created by a client connection are automatically deleted as soon as the client session ends.
Persistent node: Once created, it will not disappear as long as it is not deleted, regardless of whether the client is connected or not.
3. The principle of zookeeper
In order to ensure reliability, zookeeper cannot use one machine, but should be a cluster.
In order to ensure that the zookeeper cluster data can be consistent, there must be a person who has the final say, this is the leader, and the others are followers. There can only be one and only one leader in the cluster at a time. The leader can perform addition, deletion, modification and query operations, while the follower can only perform query operations. All update operations will be forwarded to the leader for processing, and tasks approved by the leader will be sent to followers to execute to ensure consistency with the leader.
Since the network is unstable, in order to ensure the consistency of the execution order, all tasks will be assigned a unique sequence number, and the tasks must be executed according to this number to ensure the consistency of the task sequence.
So when can the leader consider a client's request to be processed successfully?
If only the leader or a few machines approve the task, if the leader and these few machines fail, the new leader elected will not know about the task approved before, which will eventually violate the reliability of the data. Therefore, the leader is required to ensure that most of the machines in the cluster should know about the proposal before approving a task, so that even if it dies, the leader elected based on the majority agreement must know about the proposal. If the leader must wait until all followers agree to execute the proposal, it is not good, because if one machine fails, the leader cannot work, which is equivalent to a single node, and the reliability of the cluster cannot be guaranteed. Therefore, as long as more than half of the leaders pass, a proposal can be considered passed. Therefore, after receiving the task submitted by the client, the leader will send a proposal to all the followers in the cluster and wait for the follower's vote. After the followers receive the proposal, they will vote, agree or disagree, and the leader will recycle the follower's Voting, once more than half of the votes are approved, the leader considers the proposal passed, and then sends an order to require all followers to perform the tasks in the proposal.
Because of the majority consent mechanism, in the most extreme case, more than half of the machines in the cluster know the latest proposal, and if more than half of the machines fail, the remaining machines may not know the latest proposal, and there is no guarantee that the newly elected leader knows about the latest proposal. The latest proposal, so in zookeeper, the cluster must be at least half alive in order to work properly.
Summarize:
The zookeeper cluster uses the voting mechanism of more than half of the election, so it must be guaranteed that more than half of the machines survive. The number of machines in the zookeeper cluster should preferably be an odd number, because more than half of the surviving clusters are required to work, so the even number of machines The reliability of the cluster is actually the same as An even number of -1 machines provides the same cluster reliability.
The problem of leader election:
When the cluster starts at the beginning, the machine that first reaches the condition of more than half will be selected as the leader.
When the leader dies, the machine with the highest task number will be elected as the new leader by more than half of the votes. If there are the same task numbers, in the configuration file, whoever has the larger serverid is the leader.
To be continued......!