Technology sharing [zookeeper]

About zookeeper

1. What is the zookeeper

ZooKeeper is a distributed, open-source collaboration services for distributed applications.
ZooKeeper's goal is to design the complex and error-prone distributed consistency of service packages together to form a highly efficient and reliable set of primitives, and a series of easy to use interface to users.

2, zookeeper usage scenarios

2.1, data publish / subscribe, Configuration Center

1, the publisher will publish data on the node Zookeeper for subscriber data subscription.
2, Zookeeper uses a combination of push-pull mode, the client node registration they need to focus on the service side, once the node data is changed, then the server will push Watcher event notification to the appropriate client, the client receives this after the notification, the initiative to the server to obtain the latest data.

2.2, distributed lock

1, also called exclusive lock exclusive lock

① acquire the lock in needs to obtain an exclusive lock, all clients by calling the interface, create a temporary child node / exclusive_lock / lock in / exclusive_lock node. Zookeeper ensures that only one client can successfully create, without success clients need to register / exclusive_lock node listens.

② release the lock, the lock is acquired when a client down or complete normal business logic will result in the deletion of the temporary node, this time, all in / exclusive_lock node registered listening client will receive notification, you can re-initiate a distributed lock acquisition .

2, read-write lock

① When acquiring the lock, the need to acquire a shared lock, all clients are to / create a temporary order of nodes shared_lock Here, if it is a read request, then created e.g. / shared_lock / host1-R-00000001 node, if it is a write request, then create for example shared_lock / host2-W-00000002 node /.

Analyzing sequential read ②

　　　　1. After you have created a node, get all child nodes / shared_lock node, and the node changes its registered listeners.

　　　　2. Determine the sequence number in its own node in all the child nodes.

　　　　3. For a read request: if there is no child younger than their number of nodes or all smaller than their number of child nodes are read request, it indicates that they have successfully acquired shared locks, and started reading logic, if a write request , you need to wait. For a write request: if he is not a minimum number of child nodes, we need to wait.

　　　　Watcher 4. After receiving the notification, step 1 is repeated.

③ release the lock, the lock release process is consistent with the exclusive lock.

How do other kinds of locks to avoid herd behavior?

1, an exclusive lock:

2, read-write locks it?

2.2.3, a multi-coordinated from the master, master-worker

1. Use a temporary node / master represents the master. master before exercising the functions of master, we must first create the znode. If you can successfully created, start exercising master functions.
2. by creating a temporary worker nodes / workers to join the cluster below
3. master will be monitored by watch mechanism / workers of worker nodes list below to get real-time changes in worker members.

2.2.4, kafak how to use the zookeeper

1, master-worker mode
a Kafka broker cluster composed of a plurality of these systems are borker worker. Kafka will elect a worker from the controller, the controlle is the system master, responsible for the partition allocated to each topic and broker.
2, Topic registration
Topic messages are divided into a plurality of partitions and in which a plurality of distributed Broker, partition information, and the correspondence between the Broker are also maintained at the Zookeeper.
3, consumption partition relationship with consumers, consumer news displacement recorded
in the same consumer group, each partition can have only one consumer consumption, each message is consumed only once, so the partition between the consumer and the consumer relations also written zookeeper.

The important concept zookeeper

Data Model

ZooKeeper 使用文件系统模型，Datatree。Datatree 的每个节点叫作 znode。每个节点都可以保存数据。每个节点都有一个版本 (version)。版本从 0 开始计数。

基于版本号的条件更新

znode节点

1. 持久性的 znode (PERSISTENT): ZooKeeper 宕机，或者 client 宕机，这个 znode 一旦创建就不会丢失。
2. 临时性的 znode (EPHEMERAL): ZooKeeper 宕机了，或者 client 在指定的 timeout 时间内没有连接 server ，都会被认为丢失。
   znode 节点也可以是顺序性的。每一个顺序性的 znode 关联一个唯一的单调递增整数。这个单调递增整 数是 znode 名字的后缀。
3. 持久顺序性的 znode(PERSISTENT_SEQUENTIAL): znode 除了具备持久性 znode 的特点之外，znode 的名字具备顺序性。
4. 临时顺序性的 znode(EPHEMERAL_SEQUENTIAL): znode 除了具备临时性 znode 的特点之外，znode 的名字具备顺序性。

standalone 模式和 quorum模式

quorum模式要求至少3个节点

其中2181是默认客户端服务端口，3333是quorum之间的通信，3334是用于leader选举的端口。

服务器节点角色与区别

事务日志和快照

zookeeper数据一致性

1、CAP理论下，zookeeper是如何工作的。

2、zookeeper的数据一致性

• 可线性化(Linearizable)写入:先到达 leader 的写请求会被先处理，leader 决定写请求的执 行顺序。
• 客户端FIFO顺序:来自给定客户端的请求按照发送顺序执行。

3、ZAB协议

1、2PC,3PC,Paxos,Raft算法

2、ZAB（ZooKeeper Atomic Broadcast）协议

1. Leader PROPOSAL sent to all nodes in the cluster.
2. After receiving node PROPOSAL, PROPOSAL off the disc, sends an ACK to the Leader.
3. Leader after receiving ACK majority node, COMMIT is sent to all nodes in the cluster.

Leader election

1, the information contained Vote Vote: sid (server ID), zxid (transaction ID), epoch (leader cycle)

2, the voting process

By sending a ZooKeeper node to all nodes in the electoral vote to start
a node after receiving a vote if it is found received a new vote, gave his own update to the latest poll voting, vote and send to all of ZooKeeper nodes. Otherwise, do nothing.

How to tell if a vote is new?

A 3-node cluster election timing diagram of a leader:

3, split brain - a long message transmission delay results in two elected leader