一、What is ZooKeeper?
Zookeeper is a distributed , open source distributed application coordination service , an open source implementation of Google's Chubby, and an important component of Hadoop and Hbase. It is a software that provides consistent services for distributed applications. The functions provided include: configuration maintenance, domain name service, distributed synchronization, group service, etc.
Apache ZooKeeper is an effort to develop and maintain an open-source server which enables highly reliable distributed coordination.
ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. All of these kinds of services are used in some form or another by distributed applications. Each time they are implemented there is a lot of work that goes into fixing the bugs and race conditions that are inevitable. Because of the difficulty of implementing these kinds of services, applications initially usually skimp on them ,which make them brittle in the presence of change and difficult to manage. Even when done correctly, different implementations of these services lead to management complexity when the applications are deployed.
A distributed system is a system consisting of a set of computer nodes that communicate over a network and coordinate their work to accomplish a common task (a task) . The emergence of distributed systems is to use cheap and ordinary machines to complete computing and storage tasks that a single computer cannot. The purpose is to use more machines to process more data.
Second, stand-alone installation and configuration
1. Download zookeeper
wget http://mirror.bit.edu.cn/apache/zookeeper/zookeeper-3.4.11/zookeeper-3.4.11.tar.gz
2. Unzip zookeeper-3.4.11.tar.gz
tar zxvf zookeeper-3.4.6.tar.gz
3. Enter the conf directory and copy a configuration file
cd /usr/local/software/zookeeper/zookeeper-3.4.11/conf cp zoo_sample.cfg zoo.cfg
4. The content of zookeeper 's configuration file zoo.cfg is as follows
# The number of milliseconds of each tick tickTime=2000 # The number of ticks that the initial # synchronization phase can take initLimit = 10 # The number of ticks that can pass between # sending a request and getting an acknowledgement syncLimit=5 # the directory where the snapshot is stored. # do not use /tmp for storage, /tmp here is just # example sakes. dataDir=/usr/local/software/zookeeper/zookeeper-3.4.11 #log info dataLogDir=/usr/local/software/zookeeper/zookeeper-3.4.11/logs # the port at which the clients will connect clientPort=2181 # the maximum number of client connections. # increase this if you need to handle more clients #maxClientCnxns=60 # # Be sure to read the maintenance section of the # administrator guide before turning on autopurge. # # http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance # # The number of snapshots to retain in dataDir #autopurge.snapRetainCount=3 # Purge task interval in hours # Set to "0" to disable auto purge feature # autopurge.purgeInterval = 1
5. Start, stop, connect
./zkServer.sh start ./zkServer.sh stop /usr/local/software/zookeeper/zookeeper1/bin/zkCli.sh -server 127.0.0.1:2181
6.Managing ZooKeeper Storage[from http://zookeeper.apache.org/doc/current/zookeeperStarted.html]
For long running production systems ZooKeeper storage must be managed externally (dataDir and logs).
See the section on maintenance for more details.
Connecting to ZooKeeper
$ bin/zkCli.sh -server 127.0.0.1:2181
[zk: 127.0.0.1:2181(CONNECTED) 1] ls /
[zookeeper]
[zk: 127.0.0.1:2181(CONNECTED) 3] help
ZooKeeper -server host:port cmd args
stat path [watch]
set path data [version]
ls path [watch]
delquota [-n|-b] path
ls2 path [watch]
setAcl path acl
setquota -n|-b val path
history
redo cmdno
printwatches on|off
delete path [version]
sync path
listquota path
rmr path
get path [watch]
create [-s] [-e] path data acl
addauth scheme auth
quit
getAcl path
close
connect host:port
[zk: 127.0.0.1:2181(CONNECTED) 4] create /zk_test my_data
Created /zk_test
[zk: 127.0.0.1:2181(CONNECTED) 6] ls /
[zookeeper, zk_test]
[zk: 127.0.0.1:2181(CONNECTED) 7] get /zk_test
my_data
cZxid = 0x7
ctime = Mon Feb 05 23:12:33 PST 2018
mZxid = 0x7
mtime=Mon Feb 05 23:12:33 PST 2018
pZxid = 0x7
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 7
numChildren = 0
#We can change the data associated with zk_test by issuing the set command, as in:
[zk: 127.0.0.1:2181(CONNECTED) 9] set /zk_test junk
cZxid = 0x7
ctime = Mon Feb 05 23:12:33 PST 2018
mZxid = 0x8
mtime=Mon Feb 05 23:14:56 PST 2018
pZxid = 0x7
cversion = 0
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 4
numChildren = 0
#Finally, let's delete the node by issuing:
[zk: 127.0.0.1:2181(CONNECTED) 10] delete / zk_test
[zk: 127.0.0.1:2181(CONNECTED) 11] ls /
[zookeeper]
3. Cluster configuration [three servers]
There are two forms of cluster mode:
1) Use multiple machines and run one ZooKeeper Server process on each machine;2) Use one machine and run multiple ZooKeeper Server processes on this machine.
In a production environment, the first form is generally used, and in a practice environment, the second form is generally used.
I use the second method, running three ZooKeeper Server processes on one machine
1. Install three zookeepers
[root@localhost bin]# ls /usr/local/software/zookeeper/ zookeeper1 zookeeper2 zookeeper3 zookeeper-3.4.11.tar.gz
2. The zoo.cfg files of the three zookeepers
[root@localhost ~]# cat -n /usr/local/software/zookeeper/zookeeper1/conf/zoo.cfg
1 # The number of milliseconds of each tick
2 tickTime=2000
3 # The number of ticks that the initial
4 # synchronization phase can take
5
6 initLimit = 10
7 # The number of ticks that can pass between
8 # sending a request and getting an acknowledgement
9
10 syncLimit=5
11 # the directory where the snapshot is stored.
12 # do not use /tmp for storage, /tmp here is just
13 # example sakes.
14
15 dataDir=/usr/local/software/zookeeper/zookeeper1/data
16
17 #log info
18 dataLogDir=/usr/local/software/zookeeper/zookeeper1/logs
19
20 # the port at which the clients will connect
21 clientPort=2181
22 # the maximum number of client connections.
23 # increase this if you need to handle more clients
24 #maxClientCnxns=60
25 #
26 # Be sure to read the maintenance section of the
27 # administrator guide before turning on autopurge.
28 #
29 # http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
30 keeper-3.4.11
31 #
32 # The number of snapshots to retain in dataDir
33 #autopurge.snapRetainCount=3
34 # Purge task interval in hours
35 # Set to "0" to disable auto purge feature
36 # autopurge.purgeInterval = 1
37
38 #server.NUM=IP:port1:port2 NUM indicates the server number of the local machine; IP is the local ip address;
39 #port1 is the communication port between the leader and the follower; port2 is the communication port for participating in the election of the leader
40 server.1=127.0.0.1:2222:2225
41 server.2=127.0.0.1:3333:3335
42 server.3=127.0.0.1:4444:4445
[root@localhost ~]# cat -n /usr/local/software/zookeeper/zookeeper2/conf/zoo.cfg
1 # The number of milliseconds of each tick
2 tickTime=2000
3 # The number of ticks that the initial
4 # synchronization phase can take
5
6 initLimit = 10
7 # The number of ticks that can pass between
8 # sending a request and getting an acknowledgement
9
10 syncLimit=5
11 # the directory where the snapshot is stored.
12 # do not use /tmp for storage, /tmp here is just
13 # example sakes.
14
15 dataDir=/usr/local/software/zookeeper/zookeeper2/data
16
17 #log info
18 dataLogDir=/usr/local/software/zookeeper/zookeeper2/logs
19
20 # the port at which the clients will connect
21 clientPort=2182
22 # the maximum number of client connections.
23 # increase this if you need to handle more clients
24 #maxClientCnxns=60
25 #
26 # Be sure to read the maintenance section of the
27 # administrator guide before turning on autopurge.
28 #
29 # http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
30 #
31 # The number of snapshots to retain in dataDir
32 #autopurge.snapRetainCount=3
33 # Purge task interval in hours
34 # Set to "0" to disable auto purge feature
35 # autopurge.purgeInterval = 1
36
37 #server.NUM=IP:port1:port2 NUM indicates the server number of the local machine; IP is the local ip address;
38 #port1 is the communication port between the leader and the follower; port2 is the communication port for participating in the election of the leader
39 server.1=127.0.0.1:2222:2225
40 server.2=127.0.0.1:3333:3335
41 server.3=127.0.0.1:4444:4445
[root@localhost ~]# cat -n /usr/local/software/zookeeper/zookeeper3/conf/zoo.cfg
1 # The number of milliseconds of each tick
2 tickTime=2000
3 # The number of ticks that the initial
4 # synchronization phase can take
5
6 initLimit = 10
7 # The number of ticks that can pass between
8 # sending a request and getting an acknowledgement
9
10 syncLimit=5
11 # the directory where the snapshot is stored.
12 # do not use /tmp for storage, /tmp here is just
13 # example sakes.
14
15 dataDir=/usr/local/software/zookeeper/zookeeper3/data
16
17 #log info
18 dataLogDir=/usr/local/software/zookeeper/zookeeper3/logs
19
20 # the port at which the clients will connect
21 clientPort=2183
22 # the maximum number of client connections.
23 # increase this if you need to handle more clients
24 #maxClientCnxns=60
25 #
26 # Be sure to read the maintenance section of the
27 # administrator guide before turning on autopurge.
28 #
29 # http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
30 #
31 # The number of snapshots to retain in dataDir
32 #autopurge.snapRetainCount=3
33 # Purge task interval in hours
34 # Set to "0" to disable auto purge feature
35 # autopurge.purgeInterval = 1
36
37 #server.NUM=IP:port1:port2 NUM indicates the server number of the local machine; IP is the local ip address;
38 #port1 is the communication port between the leader and the follower; port2 is the communication port for participating in the election of the leader
39 server.1=127.0.0.1:2222:2225
40 server.2=127.0.0.1:3333:3335
41 server.3=127.0.0.1:4444:4445
3. Add myid file
Add the myid file to the data directory of zookeeper to store a value that is used as the identity of the ZooKeeper Server process. That is NUM in the above configuration.[root@localhost data]# cat /usr/local/software/zookeeper/zookeeper1/data/myid 1 [root@localhost data]# cat /usr/local/software/zookeeper/zookeeper2/data/myid 2 [root@localhost ~]# cat /usr/local/software/zookeeper/zookeeper3/data/myid 3
4. Start three zookeepers
[root@localhost ~]# /usr/local/software/zookeeper/zookeeper1/bin/zkServer.sh start ZooKeeper JMX enabled by default Using config: /usr/local/software/zookeeper/zookeeper1/bin/../conf/zoo.cfg Starting zookeeper ... STARTED [root@localhost ~]# /usr/local/software/zookeeper/zookeeper2/bin/zkServer.sh start ZooKeeper JMX enabled by default Using config: /usr/local/software/zookeeper/zookeeper2/bin/../conf/zoo.cfg Starting zookeeper ... STARTED [root@localhost ~]# /usr/local/software/zookeeper/zookeeper3/bin/zkServer.sh start ZooKeeper JMX enabled by default Using config: /usr/local/software/zookeeper/zookeeper3/bin/../conf/zoo.cfg Starting zookeeper ... STARTED
5. View the status of the three zookeepers
[root@localhost ~]# /usr/local/software/zookeeper/zookeeper1/bin/zkServer.sh status ZooKeeper JMX enabled by default Using config: /usr/local/software/zookeeper/zookeeper1/bin/../conf/zoo.cfg Mode: follower [root@localhost ~]# /usr/local/software/zookeeper/zookeeper2/bin/zkServer.sh status ZooKeeper JMX enabled by default Using config: /usr/local/software/zookeeper/zookeeper2/bin/../conf/zoo.cfg Mode: leader [root@localhost ~]# /usr/local/software/zookeeper/zookeeper3/bin/zkServer.sh status ZooKeeper JMX enabled by default Using config: /usr/local/software/zookeeper/zookeeper3/bin/../conf/zoo.cfg Mode: follower
6. Three zookeeper clusters
Configure multiple instances to form a cluster to provide external services to achieve the purpose of horizontal expansion. The data on each server is the same, and each server can provide external read and write services, which is the same as redis, that is Each server is equal to the client. For details on the zookeeper election mechanism, see https://www.cnblogs.com/ASPNET2008/p/6421571.html . The zookeeper cluster usually has three roles: Leader, Follower, and Observer.
