In the first reading notes , I learned about Service Registry , which is the core of the entire "microservice architecture". It not only provides Service Registry (service registration) functions, but also provides support for Service Discovery (service discovery).
Service registration refers to registering the relevant configuration information of the service in the service registry after the service is started.
Service discovery can be understood as when customers call these services, they will obtain these service configurations from the service registry through Service GateWay (service gateway), and then call specific service interfaces through reverse proxy, from the service registry. The process of getting the service configuration.
At the same time, the service registry will regularly check the registered services. If a service is found to be inaccessible, it will be removed from the service registry. This process of regular detection is called "heartbeat detection" . Therefore, the service registry has very high requirements for "distributed data consistency" , that is, once the service configuration in the service registry is changed, the notification mechanism must achieve high performance, and the service registry itself needs to be highly available. .
So, who can take on the heavy responsibility of the service registry? We consider ZooKeeper to be one of the recent solutions for service registries. Goal : Get to know ZooKeeper, learn to use ZooKeeper, and finally implement the core functions of the service registry based on ZooKeeper. At the same time, we will use Node.js to build a highly available service gateway.
1. What is ZooKeeper (What)
ZooKeeper is used to provide coordination services in a distributed environment. Developed by Yahoo using Java language, it is a sub-project of the Hadoop project and plays the role of core components in Hadoop, HBase, Kafka and other technologies. Its design goal is to encapsulate those complex and error-prone distributed consistent services to form an efficient and reliable service, and to provide users with a series of easy-to-use interfaces.
ZooKeeper is a classic distributed data consistency solution , based on which functions such as data publishing and subscription , load balancing , naming service , distributed coordination and notification , cluster management , leader election , distributed locks , and distributed queues can be implemented.
ZooKeeper generally provides external services in a cluster. A cluster contains multiple nodes, each node corresponds to a ZooKeeper server, and all nodes jointly provide external services. Including five characteristics.
- Sequential (requests sent from the same client will enter ZooKeeper in strict accordance with the order in which they are sent, similar to a queue, with a "first-in, first-out" feature)
- Atomicity (all machines in the entire cluster successfully process a request, or none of them are processed, similar to the atomicity of transactions)
- Unity (no matter which server node the client connects to, the server data model seen by the client is the same, and it is impossible to have two different data states.)
- Reliability (once the server data state changes, it will be stored immediately)
- Real-time (when a request is successfully processed, the client can immediately obtain the latest data status of the server, and the whole process is real-time)
1.1 ZooKeeper tree model
Zookeeper has a tree-like memory model, similar to a file system, with several directories and several files in each directory, but these directories and files are collectively referred to as ZNodes in ZooKeeper, and each ZNode has a corresponding path and the data it contains. The ZNode is created by the ZooKeeper client. When the client establishes a connection with the server, the server will create a Session for the client, and all the operations of the client to the ZNode are completed in this session.
| ZNode type | illustrate |
| Persistent (persistent node) | When the session ends, the node will not be deleted. |
| Persistent Sequential | When the session ends, the node will not be deleted, and the node name has an auto-increment suffix |
| Ephemeral (ephemeral node) | When the session ends, the node will be deleted |
| Ephemeral Sequential (ephemeral sequential node) | When the session ends, the node will be deleted, and the node name has an auto-incrementing monkey mouth. |
[Note] Only persistent nodes can have child nodes, which is limited by ZooKeeper.
1.2 Zookeeper cluster structure
ZooKeeper refers to the Paxos protocol and designs a more lightweight protocol called Zab (ZooKeeper Atomic Broadcast). The Zab protocol is divided into two phases: Leader Election and Atomic Broadcast . When the ZooKeeper cluster is started, one node will be elected as the leader, and the other nodes will be followers. When the leader node fails, a new leader node will be automatically elected, and all nodes will be restored to a normal state.
When the leader election phase ends, it enters the atomic broadcast phase, which will synchronize the data between the leader and each follower node to ensure that the leader and follower nodes have the same state. All write operations will be sent to the Leader node, and the data will be synchronized to other Follower nodes by broadcasting.
A ZooKeeper cluster usually consists of a set of nodes. In general, 3 to 5 nodes can form an available cluster. In theory, the more nodes the better.
2. How to use ZooKeeper (How)
Since ZooKeeper is developed based on the Java language, the JDK runtime environment needs to be installed before using it .
2.1 Running ZooKeeper
For specific steps, please refer to the article Installing ZooKeeper under Ubuntu .
2.2 Connect to ZooKeeper using command line tools
Command: bin/zkCli.sh
After a successful connection, ZooKeeper can be operated from the command line.
1) Help command to view related ZooKeeper client commands.
2) [ls] lists child nodes, [ls2] lists the basic information of child nodes and current nodes, [stat] judges whether the current path exists, [create] creates nodes, [get] obtains node data, and [set] updates nodes Data, [delete] delete the node.
Description: The -s option in the [create] command is used to specify whether the node is a sequential node, the -e option is used to specify whether the node is a temporary node, and the -acl parameter is used to control permissions; after the [set] command is executed, You can see that the dataVersion is incremented, and the dataLength attribute has changed from 5 to 2.
2.3 Connecting to ZooKeeper using the Java client
Get the jar package of Zookeeper's Java client in Maven dependencies.
<dependency> <groupId>org.apache.zookeeper</groupId> <artifactId>zookeeper</artifactId> <version>3.4.9</version> <exclusions> <exclusion> <groupId>org.slf4j</groupId> <artifactId>slf4j-log4j12</artifactId> </exclusion> <exclusion> <groupId>log4j</groupId> <artifactId>log4j</artifactId> </exclusion> </exclusions> </dependency>
The following code is used to connect to ZooKeeper.
import org.apache.zookeeper.WatchedEvent; import org.apache.zookeeper.Watcher; import org.apache.zookeeper.ZooKeeper; import java.util.concurrent.CountDownLatch; public class ZookeeperDemo { private static final String CONNECTION_STRING = "127.0.0.1:2181"; private static final int SESSION_TIMEOUT = 5000; private static CountDownLatch latch = new CountDownLatch(1); public static void main(String[] args) throws Exception { ZooKeeper zooKeeper = new ZooKeeper(CONNECTION_STRING, SESSION_TIMEOUT, new Watcher() { @Override public void process(WatchedEvent watchedEvent) { if (watchedEvent.getState() == Event.KeeperState.SyncConnected) { latch.countDown(); } } }); latch.await(); System.out.println(zooKeeper); } }
The process of establishing a ZooKeeper session is asynchronous. When the zooKeeper object is constructed, the thread will continue to execute subsequent code, but the session may not be established at this time. Therefore, you need to use the CountDownLatch tool. When the object is created, the latch.await() method is called immediately. The current thread is in a waiting state, waiting for the SyncConnected event to arrive, and then execute the latch.countDown() method. At this time, the session has been established, you can Continue to execute subsequent code.
1) List child nodes
// System.out.println(zooKeeper); // List all child nodes under the root node synchronously // List<String> children = zooKeeper.getChildren("/",null); // for (String node : children ){ // System.out.println(node); // } // List all child nodes under the root node asynchronously zooKeeper.getChildren("/", null , new AsyncCallback.Children2Callback() { @Override public void processResult(int i, String s, Object o, List<String> list, Stat stat) { for(String node: list){ System.out.println(node); } } },null);
2) Determine whether the node already exists
// Check whether there is a node in synchronization mode Stat stat = zooKeeper.exists("/", null ); if (stat != null ) { System.out.println("node exists"); } else { System.out.println("node does not exist"); } // Check whether there is a node zooKeeper.exists("/", null , new AsyncCallback.StatCallback() { @Override public void processResult(int i, String s, Object o, Stat stat) { if (stat != null) { System.out.println("node exists"); } else { System.out.println("node does not exist"); } } }, null);
3) Create a node
// Create child nodes synchronously String name = zooKeeper.create("/test", "hello" .getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT); System.out.println(name); // Create child nodes asynchronously zooKeeper.create("/test1", "hello".getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT, new AsyncCallback.StringCallback() { @Override public void processResult(int i, String s, Object o, String s1) { System.out.println(s1); } }, null);
4) Get node data
// Get node data synchronously byte [] data = zooKeeper.getData("/test", null , null ); System.out.println(new String(data)); // Get node data asynchronously zooKeeper.getData("/test1", null , new AsyncCallback.DataCallback() { @Override public void processResult(int i, String s, Object o, byte[] bytes, Stat stat) { System.out.println(new String(bytes)); } }, null);
5) Update node data
// Update node data synchronously Stat stat1 = zooKeeper.setData("/test", "hi".getBytes(), -1 ); System.out.println(stat1 != null); // Update node data asynchronously zooKeeper.setData("test1", "hi".getBytes(), -1, new AsyncCallback.StatCallback() { @Override public void processResult(int i, String s, Object o, Stat stat) { System.out.println(stat != null); } }, null);
6) Delete node data
// Delete node data synchronously zooKeeper.delete("/test", -1 ); System.out.println(true); // Delete node data asynchronously zooKeeper.delete("/test1", -1, new AsyncCallback.VoidCallback() { @Override public void processResult(int i, String s, Object o) { System.out.println(i == 0); } }, null); Thread.sleep(Long.MAX_VALUE);
ZooKeeper officially provides two sets of Java client APIs, synchronous and asynchronous.