What is ZooKeeper
ZooKeeper is a distributed, open-source collaboration services for distributed applications. ZooKeeper's goal is to design the complex and error-prone distributed consistency of service packages together to form a highly efficient and reliable set of primitives, and a series of easy to use interface to users.
ZooKeeper originated in a research group at Yahoo! Research. At the time, the researchers found that in many large internal Yahoo basic system needs to rely on a similar system for distributed collaboration, but these systems are often distributed existence of a single point.
So, Yahoo's developers to develop a common framework for coordination of distributed with no single point problem, which is ZooKeeper. ZooKeeper after being widely used in the open source community, here are three well-known open source projects is how to use ZooKeeper:
Many distributed coordination ZooKeeper service can be used to do, where typical application scenario is as follows:
ZooKeeper adapted to store critical data and related collaboration, not suitable for storing large amounts of data. If you want to keep KV or large amounts of business data, or to use a database or other NoSql do.
Why ZooKeeper not suitable for large volume data storage it? There are two major reasons:
Design: ZooKeeper all the required data (its data tree) is loaded into memory. This determines the amount of data stored ZooKeeper limited memory. This is more like ZooKeeper and Redis. Database systems such as the MySQL general (if using InnoDB storage engine) is greater than the memory may store data, since the B-Tree is based InnoDB storage engine. B-tree storage engine and LSM storage engine is greater than the amount of data can be stored in memory.
Engineering: ZooKeeper design goal is to provide data storage for the collaboration services, high availability and performance data is the most important indicators of the system, the primary objective of processing a large number of not ZooKeeper. Therefore, ZooKeeper not do too much on engineering optimization of large amount of storage.
To use the ZooKeeper service, first of all our applications to introduce ZooKeeper client library, then our client libraries and ZooKeeper clusters for network communication to use ZooKeeper service, the Client-Server architecture Essentially, our application as a client end to end calls ZooKeeper Server service.
ZooKeeper service
ZooKeeper hierarchical data model
ZooKeeper data model is a hierarchical model. Hierarchical model common in the file system. Hierarchical model and key-value data model are two mainstream models. ZooKeeper file system model based on the following two considerations:
Tree structure file system that facilitates hierarchical relationship between the expression data.
Tree structure file system that facilitates allocation of separate applications for different namespace (namespace).
The hierarchical model is referred ZooKeeper data tree. Each node Data tree called znode. Unlike the file system, each node can save data. Each node has a version (version), the version count from zero.
ZooKeeper hierarchical data model instance
data tree shown above there are two subtrees, one for the application 1 (/ app1) and another for applying 2 (/ app2).
1 sub-tree application implements a simple group membership protocol: Each client process pi create a znode p_i In / app1, long / app1 / p_i presence on behalf of a process pi in normal operation.
ZooKeeper provide external access to data tree for a simplified file system API:
UNIX-style pathname used to locate znode, for example, / A / X of znode A child node represents X.
znode only supports full amount of data read and write, not as part of the support writing and reading like a generic file system.
All API data tree is wait-free, and is being executed in the API call does not affect the completion of the other API.
API data tree of the file system are wait-free operation, the lock does not directly provide such a distributed coordination mechanism. But the data tree is very powerful API, can be used to achieve a variety of distributed collaborative mechanisms.
A znode may be permanent, or may be temporary, znode node may be of the order. Associated with each unique sequence of znode a monotonically increasing integer, there are the following four kinds ZooKeeper znode:
Persistent znode (PERSISTENT): ZooKeeper down, or client downtime, this znode once created will not be lost.
Temporary znode (EPHEMERAL): ZooKeeper is down, or the client is not connected to server within a specified timeout period, will be considered lost.
Persistence of sequential znode (PERSISTENT_SEQUENTIAL): znode includes features in addition to the persistent znode, znode name includes sequential.
Temporal order of znode (EPHEMERAL_SEQUENTIAL): znode includes features in addition to the temporary znode, znode name includes sequential.
To https://archive.apache.org/dist/zookeeper/stable/ download ZooKeeper, the current latest version is 3.5.6.
Installation ZooKeeper
The apache-zookeeper-3.5.6-bin.tar.gz extract to a local directory (the directory name is best not to include spaces and Chinese). I use / usr / local directory.
tar -zxvf apache-zookeeper-3.5.6-bin.tar.gz
The zoo_sample.cfg conf directory rename zoo.cfg, and then modify the configuration.
# Heartbeat check time of 2 seconds
tickTime = 2000
# initialization connected to the frequency and spacing server, a total time of 10 * 2 = 20 sec
10 initLimit =
frequency communication between # ZK Leader and follower, the total time of 5 * 2 = 10 seconds
=. 5 syncLimit
# stores the data snapshot in memory location, if the parameter is not set, the update transaction log is stored in the default position.
dataDir = / data / zookeeper
listening port # ZK server
clientPort = 2181
Configure the following environment variables vim / etc / profile:
export ZOOKEEPER_HOME=/usr/local/apache-zookeeper-3.5.6-bin
export PATH=$PATH:$ZOOKEEPER_HOME/bin:$ZOOKEEPER_HOME/conf
After the re-installation configuration is complete, you can start Zookeeper, use zkServer.sh start start ZooKeeper service:
[root@wupx apache-zookeeper-3.5.6-bin]# zkServer.sh start
/usr/bin/java
ZooKeeper JMX enabled by default
Using config: /usr/local/apache-zookeeper-3.5.6-bin/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
Check whether there ZooKeeper logs an error message:
[root@wupx apache-zookeeper-3.5.6-bin]# cd logs/
[root@wupx logs]# grep -E -i "((exception)|(error))" *
Because the return no results, indicating no error message.
Check the ZooKeeper data files stored here ZooKeeper transaction log file and snapshot log files.
[root@wupx zookeeper]# cd /data/zookeeper/
[root@wupx zookeeper]# tree
.
├── version-2
│ └── snapshot.0
└── zookeeper_server.pid
1 directory, 2 files
Because there is no ZooKeeper run any command, so it is not a transaction log file.
Finally, checks to see if ZooKeeper listening on port 2181.
netstat -an | ag 2181
After the execution, we can see the ZooKeeper has been listening in on 2181 this port.
Here we demonstrate how to use the next zkCli:
After the execution zkCli.sh command, there will be a lot of messages, which proves our zkCli and ZooKeeper nodes to establish a valid connection.
2019-12-22 10:38:36,684 [myid:localhost:2181] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@959] - Socket connection established, initiating session, client: /127.0.0.1:54038, server: localhost/127.0.0.1:2181
Use ls -R / recursively find znode ZooKeeper nodes, the use of create / znode_name can create znode nodes, specific presentations as follows:
# 使用 ls -R 可以递归查找 ZooKeeper 的 znode 节点
[zk: localhost:2181(CONNECTED) 0] ls -R /
/
/zookeeper
/zookeeper/config
/zookeeper/quota
# 创建 znode /app1
[zk: localhost:2181(CONNECTED) 1] create /app1
Created /app1
[zk: localhost:2181(CONNECTED) 2] create /app2
Created /app2
[zk: localhost:2181(CONNECTED) 3] create /app1/p_1 1
Created /app1/p_1
[zk: localhost:2181(CONNECTED) 4] create /app1/p_2 2
Created /app1/p_2
[zk: localhost:2181(CONNECTED) 5] create /app1/p_3 3
Created /app1/p_3
[zk: localhost:2181(CONNECTED) 6] ls -R /
/
/app1
/app2
/zookeeper
/app1/p_1
/app1/p_2
/app1/p_3
/zookeeper/config
/zookeeper/quota
Distributed lock lock holder dang if required, the lock can be released. The ephemeral ZooKeeper node just have such characteristics.
Next we come under demonstrate the need to start zkCli respectively on the two terminals,
1 on the terminal:
Execution zkCli.sh, then execute create -e / lock command to create a temporary znode, locking operation is actually established znode process, when the first lock client success.
Then try to lock the second client, on the terminal 2:
Execution zkCli.sh, then execute create -e / lock command, will find tips Node already exists: / lock, suggesting znode already exists, znode establishment fails, so lock fails, this time we have to monitor this znode, use stat -w / lock waiting for the lock to be released.
This time we exit the first client, quit command on the terminal 1, will receive a message on the client WATCHER 2, as follows:
WATCHER::
WatchedEvent state:SyncConnected type:NodeDeleted path:/lock
After you receive this event again in the client 2 performs lock, perform create -e / lock, will display create znode success, success that is locked.
This article describes the installation configuration ZooKeeper basic concepts ZooKeeper and zkCli use, and use zkCli to achieve a lock for the back of more in-depth study to lay the foundation.