1. Introduction
1.1 What is it?
Zookeeper is an open source distributed Apache project that provides coordination services for distributed applications
1.2, design goals
- Zookeeper is not used for development and most of them provide services for other clusters
- Most applications need to develop private coordination programs, but repeatedly writing coordination programs requires a lot of time, manpower and other resources. In order to have a common coordination service mechanism , Zookeeper was born.
- The design goal of ZooKeeper is to encapsulate those complex and error-prone distributed consistency services to form an efficient and reliable primitive set , and provide users with a series of simple and easy-to-use interfaces .
Primitive: Operating system or computer network terminology category. It is composed of several instructions, used to complete a process of certain functions. It is indivisible, that is, the execution of the primitive must be continuous, and it is not allowed to be interrupted during the execution.
1.3, role
Zookeeper can be used to implement functions such as data publishing/subscription , load balancing , command service , distributed coordination/notification , cluster management , master election , distributed locks, and distributed queues that are common in distributed systems .
2. Features
- A cluster composed of a leader (Leader) and multiple followers (Follower)
LeaderResponsible for the initiation and resolution of voting, and update the system status.
FollowerUsed to receive customer requests and return results to the end, and participate in voting during the leader election process - As long as more than half of the nodes in the cluster survive, the Zookeeper cluster can serve normally
- The global data is consistent. Each server saves a copy of the same data , no matter which server the Client connects to, the data is consistent
- Update requests are carried out in sequence , and the update requests from the same Client are executed in the order in which they are sent
- The atomicity of data update, a data update either succeeds or fails
- Real-time , within a certain time range, Client can read the latest data
Three, data structure
Zookeeper maintains a data structure similar to a file system . Each node is called a Znode. We can add and delete Znodes freely, and add and delete sub-Znodes under a Znode. Unlike other file systems, Znodes can store data .
-
Znode
■ Each Znode can store by default1MBData
■ Every Znode can pass itsUnique pathLogo -
Znode node type
■ PERSISTENT: Persistent node, the default type
■ After the client disconnects from zookeeper, the node still exists
■ PERSISTENT_SEQUENTIAL: Persistent sequence number node
■ After the client disconnects from zookeeper, the node still exists, but Zookeeper numbers the node name in sequence
■ EPHEMERAL: Temporary node
▪ After the client disconnects from zookeeper, the node is deleted
■ EPHEMERAL_SEQUENTIAL: Temporary sequence numbering node
▪ After the client disconnects from zookeeper, the node is deleted, but Zookeeper serially number the node name
4. Working mechanism
4.1, working principle
Zookeeper = file system + notification mechanism
■ Zookeeper is a distributed service management framework designed based on the observer mode from the design mode. It is responsible for storing and managing the data that everyone cares about , and then accepting the observer's registration.
■ Once the data is in the state When a change occurs, Zookeeper will be responsible for notifying those observers who have been registered on Zookeeper to respond accordingly, so as to achieve a similar Master/Slave management mode in the cluster
4.2. Election mechanism
Reference with a large vernacular explain to you the election mechanism Zookeeper
4.2.1. Basic concepts
- Half mechanism: More than half of the machines in the cluster survive and the cluster is available. So zookeeper is suitable to be installed on an odd number of machines.
- Although Zookeeper does not specify master and slave in the configuration file. However, when zookeeper works, one node is the leader and the other is the follower. The leader is temporarily generated through the internal election mechanism.
- As for the leader selection mechanism, zookeeper provides three ways:
▶LeaderElection
▶AuthFastLeaderElection
▶FastLeaderElection
4.2.2, FastLeaderElection election process
When one of the following two situations occurs on a server in the Zookeeper cluster, you need to enter Leader 选举
(1) The server is initialized and started
- Server 1 starts, votes for itself, and then sends voting information. Since other machines have not started yet, it cannot receive feedback information. The status of server 1 always belongs to
LOOKING(election status). - Server 2 starts, vote for yourself, and exchange results with the previously started server 1. Server 2 wins because the number of server 2 is large, but at this time the number of votes is not more than half, so the status of the two servers is still the same
LOOKING. - Server 3 starts, vote for yourself, and exchange information with the previously started server 1 and 2. As the number of server 3 is the largest, server 3 wins. At this time, the number of votes is more than half. The status of server 1 and 2 is changed to
FOLLOWINGthat of server 3, and the status of server 3 is changed. The status isLEADING. - Server 4 starts, vote for yourself and exchange information with the previous servers 1, 2, and 3. Although the number of server 4 is large, server 3 has won before, so server 4 can only become a kid.
- Server 5 starts, and the following logic is the same as that of server 4 to become a younger brother
In the end, server 3 is the Leader, and the status is LEADING; the rest of the servers are Followers, and the status isFOLLOWING
(2) Leader failure during server operation
Cluster leader node failure The
operation period election is basically similar to the initial state voting process, which can be roughly divided into the following steps:
(1) Status change. After the Leader fails, the remaining non-Observer servers will change their server status to LOOKING, and then begin the Leader election process.
(2) Each Server will issue a vote.
(3) Receive votes from each server, and change the vote if the data of other servers is newer than your own.
(4) Processing and counting votes. After each round of voting, votes will be counted, and more than half of them can be elected.
(5) Change the status of the server and announce the election.
Without further ado, let's first take a picture: the
election process after the runner leader fails
(1) When voting for the first time, each machine will vote for itself.
(2) Then each machine will send its vote to other machines. If you find that the zxid of other machines is larger than itself, you need to change your vote and vote again. For example, server1 received three votes and found that server2's xzid was 102, pk found out that he had lost, and then decided to vote for server2 as the boss.
4.2.3, core concepts
(1) Server id (or sid): server ID
For example, there are three servers, the numbers are 1, 2, and 3. The larger the number, the greater the weight in the selection algorithm. For example, the comparison is performed based on the server ID during initial startup.
(2) Zxid: transaction ID
The transaction ID of the data stored in the server. The larger the value, the newer the data. In the election algorithm, the newer the data, the greater the weight.
(3) Epoch: logic clock
Also called the number of votes, the logical clock value in the same round of voting is the same, and this data will increase every time a vote is cast.
(4) Server status: election status
LOOKING, Campaign status
FOLLOWING, Follower status, synchronizing leader status, participating in voting
OBSERVING, Observe state, synchronize leader state, not participate in voting
LEADING, Leader status
4.2.4. Summary
(1) Zookeeper election will occur in the initial state and running state of the server.
(2) In the initial state, it will be compared according to the server sid number. The larger the number, the greater the weight, and the leader can be selected if more than half of the votes are cast.
(3) Leader failure will trigger a new round of elections. The newer zxid represents the data, the greater the weight.
(4) During the running election, there may be a split-brain situation. You can learn by yourself.
4.3. Listener principle
(1) First, there must be a main() thread
(2) Create Zookeeper client in the thread, then two threads will be created, one is responsible for network connection communication (connet), the other is responsible for listening (listener)
(3) Pass The connect thread sends the registered listener event to Zookeeper
(4) Add the event to the list of registered listeners in Zookeeper
(5) When Zookeeper listens to data or path changes, it will send this message to the listener thread
(6) ) The process() method is called inside the listener thread for processing
. 2. Common monitoring
(1) Monitoring changes in node data:
get path [watch]
(2) Monitoring changes in child nodes increase and decrease
ls path [watch]
4.4. Work flow
(1) The client sends a request to write data to the zookeeper server
(2) If the server is not a leader, it will forward the request to the leader
(3) the leader processes the request, broadcasts data to all followers, and the follower executes the write Operation
(4) As long as half of the followers feedback OK, the leader will send a commit message to all the followers
