Xiaobai Getting Started Guide | Zookeeper Quick Start

Getting started with zookeeper

Overview

Open source, distributed, Apache project for distributed applications

Working Mechanism

Zookeeper understands from the perspective of design patterns: it is a distributed service management framework designed based on the observer pattern. It is responsible for managing storage and managing the data that everyone cares about, and then accepts the observer's registration. Once the state of these data changes, zookeeper Will be responsible for notifying those observers who have registered with zookeeper to respond accordingly

characteristic

• Global data consistency
• reliability
• Sequential
• Data update atomicity
• real-time

zookeeper role

• Leader

The core of the Zookeeper cluster is
the only scheduler and processor of transaction requests (write operations), ensuring the sequence of transactions in the
cluster ; the scheduler of each server in the cluster.
For create, setData, delete and other write requests, they need to be forwarded to the leader for processing. The leader needs to determine the number and perform the operation. This process is called a transaction.

• Follower

Process client non-transaction requests, forward transaction requests to Leader;
participate in cluster Leader election.

• Observer

Observer role, observe the latest state changes of the zookeeper cluster and synchronize these states. It can handle non-transactional requests separately, and forward the transaction requests to the leader for processing. It
does not participate in any form of voting, and only provides non-transactional services.

Zookeeper cluster construction

Build steps

• Configure host name to IP address mapping
• Modify the zookeeper configuration file
• Remotely copy and distribute installation files
• Set myid
• Start zookeeper cluster

Build process

1. Download zookeeper wget http://mirror.bit.edu.cn/apache/zookeeper/zookeeper-3.4.13/zookeeper-3.4.13.tar.gz
2. Unzip it to /usr/local/src under tar -xvzf zookeeper-3.4.13.tar.gz
3. Renamed mv zookeeper-3.4.13 zookeeper
4. Enter into the conf directory
5. Copy three copies of zoo_sample.cfg and name them as zoo_1.cfg zoo_2.cfg zoo_3.cfg
6. Modify The content of zoo_1.cfg is as follows

tickTime=2000
initLimit=10
syncLimit=5
dataDir=/usr/local/data/zk1/data
clientPort=2181
server.1=localhost:2287:3387
server.2=localhost:2288:3388
server.3=localhost:2289:3389

7. Modify the content of zoo_2.cfg as follows

tickTime=2000
initLimit=10
syncLimit=5
dataDir=/usr/local/data/zk1/data
clientPort=2182
server.1=localhost:2287:3387
server.2=localhost:2288:3388
server.3=localhost:2289:3389

8. Modify the content of zoo_3.cfg as follows

tickTime=2000
initLimit=10
syncLimit=5
dataDir=/usr/local/data/zk1/data
clientPort=2183
server.1=localhost:2287:3387
server.2=localhost:2288:3388
server.3=localhost:2289:3389

9. Execute the command to create the folder and myid

mkdir -p /usr/local/data/zk1/data
mkdir -p /usr/local/data/zk2/data
mkdir -p /usr/local/data/zk3/data
echo "1" > /usr/local/data/zk1/data/myid
echo "2" > /usr/local/data/zk2/data/myid
echo "3" > /usr/local/data/zk3/data/myid

10. Execute the following command in the bin directory to start the zookeepe cluster

zkServer status ../conf/zoo_1.cfg
zkServer status ../conf/zoo_2.cfg
zkServer status ../conf/zoo_3.cfg

11.Execute the following commands in the bin directory to view the status of each node

zkServer.sh status ../conf/zoo_1.cfg
zkServer.sh status ../conf/zoo_2.cfg
zkServer.sh status ../conf/zoo_3.cfg

zookeeper data model

• Znode has both file and directory features

•   即像文件一样维护着数据又像目录一样可作为路径标识的一部分
  

• Znode has atomic operation
• Znode storage has a limit of kb level
• Znode is referenced by path, and the path must be absolute
• Znoe composition
  1. stat status information, describing the version and permission information of the Znode
  2. data The data associated with the Znode
  3. children the child nodes under the Znode
• Node type
  1. The session of the temporary node ends, the temporary node is automatically deleted, and the temporary node is not allowed to have child nodes
  2. Permanent nodes disappear unless manually deleted and cannot be modified into temporary nodes
  3. Serialize node
• Node attributes
  1. dataversion will modify dataversion after each set
  2. Cversion will be updated after the cversion child node is modified
  3. czxid Transaction ID created by Znode
  4. mzxid The transaction ID that was modified
  5. ctime node creation time
  6. mtime node update time
  7. ephemeralOwner If the node is a temporary node, the value identifies the sessionid bound to the node, if not, it is 0

zookeeper command

• Create connection

zkCli.sh -p host:port

• Create node

create [-e] [-s] path acl
-e 创建临时节点 -s 创建序列化节点

• Delete node

delete path
不能删除有子节点的节点，除非子节点内容为空

• Recursively delete nodes

rmr path

• Node limit

setquota -n | -b val path
n 标识子节点个数 -b 表示数据大小
listquota 查看设置信息
delquota -n | -b path 删除设置信息

• History command

history 查看历史命令
redo num 重新执行某条命令   

Zookeeper Watcher

Zookeeper provides distributed data publish/subscribe function. Zookeeper introduces watcher mechanism to realize this distributed notification function. Zookeeper allows the client to register a watcher to the server. When some events on the server trigger the watcher, then it will Want the client to send notifications.
Types of trigger events: node deletion, node change, child node change, etc.
In general, watcher is summarized as the following three processes. The client registers the watcher with the server, the server event triggers the watcher, and the client calls back the watcher to get the content of the trigger event.

Trigger mechanism characteristics

• One-time trigger
• Event encapsulation

•   传递watchedevent,包括通知状态，事件类型，节点路径。
  

• Asynchronous event sending
• Register first and then trigger

Shell operation watcher

help 查看所有命令，后面带有watcher的都可以实现监听

	stat path [watch]
	ls path [watch]
	ls2 path [watch]
	get path [watch]

Zookeeper java api

• Introduce dependencies

    <dependency>
      <groupId>org.apache.zookeeper</groupId>
      <artifactId>zookeeper</artifactId>
      <version>3.4.14</version>
    </dependency>

• Sample code

    public static void main(String[] args) throws Exception{
        ZooKeeper zooKeeper = new ZooKeeper("192.168.195.100:2181,192.168.195.100:2182", 30000, new Watcher() {
            @Override
            public void process(WatchedEvent watchedEvent) {
                System.out.println("回调通知");
                System.out.println(watchedEvent.getPath());
                System.out.println(watchedEvent.getState());
                System.out.println(watchedEvent.getType());
            }
        });
//        zooKeeper.create("/magicbook","充满魔力的book".getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT_SEQUENTIAL);
        zooKeeper.getData("/magicbook0000000007",true,null);
        zooKeeper.setData("/magicbook0000000007","少时诵诗书".getBytes(),-1);
        zooKeeper.delete("/magicbook0000000007",-1);
        //        zooKeeper.close();
    }

Zookeeper election mechanism

Zookeeper's default election algorithm is fastleaderelection, which uses more than half of the votes to win

concept

• Server id

•   id越大，在选举算法中权重越大
  

• Election status
  1. Campaign status LOCKING
  2. Follower status FOLOWING Synchronize leader status to participate in voting
  3. Observation status OBSERVER synchronization Leader status does not participate in voting
  4. Leader status
• 
  The version of the latest data stored in the data ID server, the larger the value, the newer the data

New cluster election

• Every machine vote for itself LOCKING
• The server's big ID can win the small ID votes
• More than half of the votes are over
• At this time, if a larger server ID comes to the election, I'm sorry. Voting is over

Non-new cluster

For a zookeeper cluster that is running normally, there is a machine downtime in the middle, and the election needs to be re-elected. The data ID, server ID, and logical clock need to be added to the election process.

• Data ID: The new version of the data is bigger, and the data will update the version every time
• Server ID: is the myid we configured
• This value starts from 0, and each election corresponds to a value. If in the same election, this is the same.
• The process of election
  1. The logical clock is ignored, and the vote is re-voted.
  2. After unifying the logic clock, the data ID wins.
  3. In the case of the same data ID, the server with the larger ID wins.
  4. Finally choose the leader

typical application

Data publishing and subscription

By repeatedly monitoring node change events, using watcher to monitor, realize data publishing and subscription

Distributed lock

1. Each client creates a temporary ordered node.
2. The client obtains the node list and judges whether it is the first node in the list. If it is the first node in the list, it obtains the lock. If it is not, it monitors the previous node and waits for the previous node to be deleted. .
3. If the lock is acquired, the normal business process is performed, and the lock is released after the execution.
In the above step 2, some people may be worried that if the node finds that it is not the node with the smallest sequence, it is ready to add a listener, but at this time the previous node is just deleted. Adding a listener at this time will never work. In fact, the zk API can guarantee Reading and adding listeners is an atomic operation.
Why listen to the previous node instead of all nodes? This is because if you monitor all child nodes, then any child node status changes, all other child nodes will receive notifications (herd effect), and we only want the next child node to receive notifications.