Zookeeper（一）

introduce

Distributed collaboration. Zookeeper is an open source implementation of Google's chubby project. It was originally a sub-project of hadoop.

Xiaomi Mi Chat and Taobao Taokeeper are actually similar to zookeeper.

Kafka uses zookeeper. Message publish and subscribe, where zk is used to detect node crashes. Implement topic discovery, and maintain topic production and consumption status.

Hbase three-stage search, Root-Region=”Meta Region=”Region (Table). The metadata information of hbase is placed in HBase. HMaster hangs up, and the node needs to be restored immediately.

Hadoop. NameNode (SecondaryNameNode), HA Hadoop. In general, a simple hadoop cluster has only one NameNode. If the NameNode hangs, the hadoop cluster is unavailable. zk is used in HA Haoop.

 

solved problem

1. Master node management. The master is highly available (who will be responsible for the work after it hangs up) and is guaranteed to be unique.
2. Profile management. The configuration files are stored in zk and distributed by zk.
3. Publish and subscribe. The publisher (producer) publishes data to the zk node for subscribers (consumer dynamic acquisition).
4. Distributed lock. The distributed environment accesses the same resource, which is implemented by third-party locks. Coordinated by zk to ensure data consistency.
5. Management of the cluster. Worker cluster monitoring.

 

installation and configuration

1. Download

　　wget http://mirror.bit.edu.cn/apache/zookeeper/zookeeper-3.4.9/zookeeper-3.4.9.tar.gz

2. Decompression

　　tar -xzvf zookeeper-3.4.9.tar.gz

3. Modify the configuration

　　../conf/zoo_sample.cfg

# Session time in milliseconds
tickTime = 2000

# Master and slave communication time, in seconds
initLimit = 10

# Master and slave heartbeat detection time, in seconds
syncLimit=5

# Save ZK snapshot and data
dataDir=/usr/zookeeper/data/zk1

# client port
clientPort=2181

# service.1 represents the server, the myid file needs to be created in dataDir, the content is 1; localhost represents the master server; the first port represents the communication port; the second port represents the election port
service.1=localhost:2888:3888
service.2=localhost:2889:3889
service.3=localhost.2890:3890

 

Common API

start up

[root@localhost conf]# zkServer.sh start /usr/local/zookeeper-3.4.6/conf/zoo1.cfg

stop

[root@localhost conf]# zkServer.sh stop /usr/local/zookeeper-3.4.6/conf/zoo1.cfg

View status

[root@localhost conf]# zkServer.sh status /usr/local/zookeeper-3.4.6/conf/zoo1.cfg

 

1.      Master node management