ZooKeeper is a distributed coordination service for Hadoop that can be used to build general distributed applications.
ZooKeeper can be thought of as a file system with high availability characteristics. This file system uses node znodes to group into a tree-like hierarchy. A znode can store data and associated ACLs, and the data stored by a znode is limited to 1MB.
ZooKeeper's data access is atomic. When reading znode data, either all the data is read or the read fails; the same is true for the write operation, either the write succeeds or the write fails, and there is no partial write success.
ZooKeeper is referenced by path. For example /zoo/cat.
The type of a znode cannot be modified after creation.
There are exactly four types of znodes:
1. Ephemeral znode: The client that created the ephemeral znode will delete it after the session ends.
2. Ephemeral sequential znodes: Ephemeral znodes have sequence numbers added.
3. Persistent znode: It will only be deleted when the client explicitly wants to delete it.
4. Persistent sequential znodes: Persistent znodes have sequential numbers added.
Sequence numbers allow global ordering of all events, so that clients can infer the order of events based on sequence numbers. Shared locks can be implemented by sequence numbers.
The observation mechanism of ZooKeeper, when the znode changes in some way, the observation mechanism (watcher) allows the client to be notified. But observations can only be triggered once, and in order to achieve multiple notifications, the required observations need to be re-registered.
There are 9 basic operations in ZooKeeper:
create: Create a znode, which must have a parent node
delete: delete a znode
exists: Does the znode exist and returns metadata
getChildren: Get a list of child nodes of a znode
getData: Get the data saved by the znode
setData: Modify the data saved by the znode
getACL: Get the ACL of a znode
setACL: modify the ACL of a znode
sync: synchronize the client's view with ZooKeeper's
The update operation in ZooKeeper is conditional, delete and setData need to provide a version number, and the version number can be executed successfully.
There is an operation called multi in ZooKeeper, which is used to aggregate multiple operations into a single operation unit. A transaction-like function can ensure that these sub-operations either all succeed or all fail.
ZooKeeper can set observations on exists/getData/getChildren. Watches can be triggered by write operations create/delete/setData, and watch events are generated when the watch is triggered.
| Watch the trigger | |||||
| Set the action for observation | create znode | Create child nodes | delete znode |
delete child node | setData |
| exists | NodeCreated | NodeDelete | NodeDat- aChanged |
||
| getData | NodeDelete | NodeDat- aChanged |
|||
| getChildren | NodeChildr- enChanged |
NodeDelete | NodeChildre- nChanged |
An ACL list is created when a znode is created, and ZooKeeper provides three authentication mechanisms
1.Digest identifies the client by username and password. zk.addAuthInfo("digest","user:password".getBytes())
2.sasl identifies clients through Kerberos
3.IP Identify the client by the client Ip address. new ACL(Perms.READ,new IP("ip","10.0.0.1"))
ACL permission list
| ACL permissions | allowed operations |
| create | create child node |
| read | getChildren/getNode |
| write | setData |
| delete | delete child node |
| admin | setACL |
ZooKeeper supports third-party authentication system insertion.
The ZooKeeper service has two different operating modes:
1. Standalone mode
2. Copy mode
ZooKeeper achieves high availability through replication, providing service as long as more than half of the ensemble is available (this is why an odd number of servers is recommended).
ZooKeeper uses the Zab protocol, which consists of two stages that can be repeated indefinitely
1. Leader election
2. Atomic broadcast All write requests are forwarded to the leader, and the leader broadcasts the update to the followers. The leader will only submit the update when more than half of the followers have modified it.
Consistency provided by the ZooKeeper service:
1. Sequential consistency The client changes the value of znode to a and then to b, no client can see b first and then a
2. Atomicity Each update operation either succeeds or fails
3. Single system image A client connecting to which server sees the same system view.
4. Persistence Once the update is successful, the result will persist and will not be revoked
5. Timeliness When the client reads the value of the znode, it executes the sync operation to get the latest value
Every update to the znode tree is given a globally unique ID called zxid. ZooKeeper requires all updates to be numbered and ordered, which determines the order of execution in the distributed system.
A list of servers in the ensemble is included in the configuration of each ZooKeeper client.
After the client connects to the ZooKeeper server, each session will have a timeout set.
The client keeps the session from expiring via ping requests (heartbeats).
Clients can automatically failover.
Short session timeouts will detect machine failures faster, but may experience vibration (servers repeatedly leaving and then rejoining for a short period of time).
For applications that create more complex transient state, longer session timeouts should be set.
The more servers in the ZooKeeper ensemble, the larger the session timeout setting should be.
ZooKeeper can implement a distributed lock service. First, specify a znode as a lock. Clients wishing to acquire the lock create some znodes in short order, and the lock is acquired with the smallest sequence number.