Article Directory
Recommended Reading
- What is the distributed coordination technology?
- What is a distributed lock?
- How to achieve Zookeeper distributed lock?
- Zookeeper build environment Linux environment
- ZooKeeper interview questions (latest edition 2020)
What is ZooKeeper?
ZooKeeper is a distributed coordination service for managing mainframe. In a distributed environment, coordination and management services is a complex process. ZooKeeper solve this problem through its simple architecture and API. ZooKeeper allows developers to focus on core application logic without having to worry about the distributed nature of applications.
Common services provided by ZooKeeper
- Naming Service - by name identifies the nodes in the cluster. It is similar to DNS, but only for the node.
- Configuration management - adding nodes recent and latest system configuration information.
- Cluster Management - in real-time joining / leaving nodes in the cluster and node status.
- Election algorithm - a node election as leader coordination purposes.
- Locking and synchronization service - lock data while modifying data. This mechanism can help you automatically failover when connected to other distributed applications (such as Apache HBase).
- Registry highly reliable data - data can be obtained even when turned off in one or several nodes.
Zookeeper data model
- It is much like the data among the trees, but also very much like a file system directory.
- Tree is composed of nodes, the Zookeeper data storage node is also based, this node is called Znode. However, the node is different from the tree, the path reference znode of reference, similar to a file path . Such a hierarchical structure, so that each node has a Znode the only path, just as the namespace make clear isolation of different information.
Znode which elements contained
- data: data storage Znode.
- ACL: access Znode record, that is who or what can access this IP node.
- stat: Znode containing various metadata, such as transaction ID, version number, timestamp, size, etc.
- child: a reference to the current node's children
Note: Zookeeper for reading and writing little scene design. Znode is not used to store massive business data, but are used to store a small amount of status and configuration information, the maximum data for each node can not exceed 1MB.
Znode type
- Znode is divided lasting (persistent) node, the sequence (Sequential) node and the temporary (ephemeral) node.
- Persistent node - even after the creation of this particular znode client disconnects, lasting node still exists. By default, unless otherwise indicated, all znode are persistent.
- Temporary node - client active, temporarily node is valid. When the client is disconnected with ZooKeeper collection, temporary node will be automatically deleted. Therefore, only temporary node does not allow children. If the temporary node is deleted, the next appropriate node to its filling position. Interim leader node plays an important role in the election.
- Order node - the node order can be permanent or temporary. Znode When a new node is created as a sequence, ZooKeeper by 10-bit serial number appended to the original name set znode path. For example, if you create a path / myapp znode of a sequential node, then the path will change ZooKeeper / myapp0000000001, and the next sequence number to 0,000,000,002. If two sequential nodes are created at the same time, it will not be the same for each ZooKeeper znode using digital. In order node locking and synchronization play an important role.
ZooKeeper architecture
- ZooKeeper of "client - server architecture."
- Each component is described in
| section | description |
|---|---|
| Client | The client, a node in the cluster our distributed applications, and access information from the server. For a particular time interval, each client sends a message to the server so that the server knows the client is active. Similarly, when the client is connected, the server sends confirmation code. If the connected server does not respond, clients will automatically redirect the message to another server. |
| Server | Server, our overall a node in ZooKeeper, provides clients with all the services. Sends a confirmation code to tell the server to the client is active. |
| Ensemble | ZooKeeper server group. Minimum number of nodes is 3 to form the desired ensemble. |
| Leader | Server node, if any connected node fails, the automatic recovery. Leader is elected when the service starts. |
| Follower | Follow the leader instruction server node. |
The basic operation of Zookeeper
- Creating nodes
create - Delete Node
delete - Determine whether there is a node
exists - Obtaining data of a node
getData - Setting data of a node
setData - Gets all the child nodes of node
getChildren
exists, getData, getChildren read operation belongs.
- Zookeeper client when requesting a read operation, can select whether to set Watch.
Watches (monitoring)
- Monitoring is a simple mechanism that allows the client to receive notification of ZooKeeper set to change. The client may be provided at the time of reading a particular Watches znode. Watches will send any znode (client registry) to registered clients notified of the change.
- Znode change is a child or znode modify data associated with znode the changes. Only trigger once watches. If the client wants to notice Again, it must be done by another read operation. When the connection session expires, the client will be disconnected from the server, related watches will also be deleted.
Sessions (sessions)
- ZooKeeper session is very important for the operation. Requesting session performed in FIFO order. Once the client connects to the server, the client will establish a session and allocate a session ID.
- The client send heartbeat interval at specific time effective to maintain the session. During the (session timeout) specified when the server is turned over if ZooKeeper collections are not received from the client heartbeat, it will determine the client crashes.
- Session timeout typically in milliseconds. When the session ends for any reason, temporary node created during that session will be deleted.
Zookeeper workflow
- Once ZooKeeper collection starts, it waits for client connections. The client will connect to a node ZooKeeper set. It can be a leader or a follower node. Once the client is connected, the nodes will be assigned a particular client session ID to the client sends an acknowledgment. If the client does not receive an acknowledgment, it tries to connect to another node ZooKeeper collection. Once connected to the node, the client will send a heartbeat at regular intervals to the node to ensure that connections are not lost.
- If the client wants to read a particular znode, it will send a read request to the node having znode path and node acquires its own database to return the requested znode. To this end, read speed ZooKeeper set quickly.
- If the client wants to set in ZooKeeper, it will be transmitted to the data path and znode server data store. The server forwards the request to connect leader, leader and follower will reissue all of the write request. If only the most successful response node, the write request is successful, a successful return code will be sent to the client. Otherwise, the write request fails. The vast majority of nodes are called Quorum.
| Package | description |
|---|---|
| Write (write) | Write process handled by the leader node. leader will forward the request to write all znode, and wait for the reply znode. If half of the znode reply, the writing process is complete. |
| Read (read) | Reading is performed by a specific connection znode inside, there is no need to interact with the cluster. |
| Copy the database (replicated database) | It is used to store data in the zookeeper. Each znode has its own database, each znode in conformance help each have the same data. |
| Leader | Leader is responsible for handling a write request Znode. |
| Follower | follower write request received from a client, and forwards them to the leader znode. |
| Request processor (request processor) | It exists only in the leader node. It manages the write request from the follower node. |
| Broadcast atom (atomic broadcasts) | Responsible for changes in broadcasting from leader to follower node node. |
Zookeeper consistency
Zookeeper as a distributed system coordination services, if their hang of how to deal with it?
- In order to prevent a stand-alone hang, Zookeeper maintains a cluster. As shown below:
- Zookeeper Service cluster is a master multi-slave structure.
- When updating data, the first update to the master node (here refers to a server node, not znode), re-synchronization to the node.
- When reading data, read directly from any node.
- To ensure data consistency from the master node, using the Zookeeper ZAB protocol that is very similar to and consistent Paxos algorithm Raft.
Zookeeper application scenarios
Distributed Lock
- This is a design researcher at Yahoo Zookeeper mind. Zookeeper node using the temporary order can be easily distributed lock.
Service registration and discovery
- Use Znode and Watcher, you can achieve distributed registration and discovery services. The most famous application is distributed RPC framework Dubbo Ali.
Sharing configuration and status information
- CODIS Redis distributed solutions, on the use of meta-information Zookeeper to store data, and routing table codis-proxy node. Meanwhile codis-config command will launch synchronized to each surviving codis-proxy through ZooKeeper.
- In addition, Kafka, HBase, Hadoop, Zookeeper also rely on synchronization node information to achieve high availability.
The more you know, the more you do not know.
Proper way without surgery, patients can still seek, there is no way to surgery, ending surgery.
If you have other questions, welcome message, we can discuss, learn together and progress together
