Acquisition: 100,000-word interview cheat sheet
During the interview, as long as the interviewer sees that Zookeeper (familiar and master) is written on your resume, then you must be prepared for the next 11 consecutive questions.
NO1: What is zookeeper?
ZooKeeper is a distributed, open source distributed application coordination service. It is an open source implementation of Google’s Chubby (Chubby is not open source). It is the manager of the cluster and monitors the status of each node in the cluster. Submit the feedback for the next reasonable operation. In the end, a simple and easy-to-use interface and a system with high performance and stable functions will be provided to users.
One of the most common usage scenarios of Zookeeper is to serve as the registration center for service producers and service consumers. Service producers register their services to the Zookeeper center, and service consumers first look in Zookeeper when making service calls Service, after obtaining the detailed information of the service producer, call the content and data of the service producer. A simple example is as follows:
NO2: Do you know the system architecture of Zookeeper?
The main things we need to understand and master in the architecture diagram of Zoo Keeper are:
(1) ZooKeeper is divided into server (Server) and client (Client). The client can connect to any server of the entire ZooKeeper service (unless the leaderServes parameter is explicitly set and the leader is not allowed to accept client connections).
(2) The client uses and maintains a TCP connection through which it sends requests, receives responses, obtains observed events, and sends heartbeats. If this TCP connection is interrupted, the client will automatically try to connect to another ZooKeeper server. When a client connects to the ZooKeeper service for the first time, the ZooKeeper server that accepts the connection will establish a session for the client. When this client connects to another server, the session will be re-established by the new server.
(3) Each Server in the above figure represents a machine where Zookeeper service is installed, that is, the entire cluster that provides Zookeeper service (or is composed of pseudo clusters);
(4) The servers that make up the ZooKeeper service must know each other. They maintain a state image in memory, as well as transaction logs and snapshots in persistent storage. As long as most servers are available, the ZooKeeper service is available;
(5) When ZooKeeper is started, a leader will be elected from the instance, and the leader is responsible for processing data updates and other operations. A sign of a successful update operation is if and only if most servers successfully modify data in memory. Each server stores a copy of data in memory.
(6) Zookeeper can be replicated in clusters, and data consistency is maintained between clusters through the Zab protocol (Zookeeper Atomic Broadcast);
(7) The Zab protocol includes two phases: the leader election phase and the Atomic Brodcast phase.
- a) A leader is elected in the cluster, and other machines are called followers. All write operations are sent to the leader, and all updates are notified to the follower through brodcast.
- b) When the leader crashes or the leader loses most of its followers, a new leader needs to be re-elected to restore all servers to a correct state.
- c) When the leader is elected, and most servers have completed synchronization with the leader's state, the leader election process is over, and the process of Atomic brodcast will be entered.
- d) Atomic Brodcast synchronizes the information between leader and follower to ensure that the leader and follower have the same system status.
NO3: Can you talk about the working principle of Zookeeper?
The core of Zookeeper is atomic broadcasting. This mechanism ensures synchronization between servers. The protocol that implements this mechanism is called the Zab protocol.
The Zab protocol has two modes, which are recovery mode (primary selection) and broadcast mode (synchronization).
The full name of the Zab protocol is Zookeeper Atomic Broadcast** (Zookeeper Atomic Broadcast). Zookeeper uses the Zab protocol to ensure the final consistency of distributed transactions. The Zab protocol requires each leader to go through three stages: discovery, synchronization, and broadcast.
When the service starts or after the leader crashes, Zab enters the recovery mode. When the leader is elected and most of the servers are synchronized with the leader's state, the recovery mode ends. State synchronization ensures that the leader and server have the same system state.
In order to ensure the consistency of the transaction sequence, zookeeper uses an increasing transaction id number (zxid) to identify the transaction. All proposals add zxid when they are made. In the implementation, zxid is a 64-bit number. Its high 32 bits are used by the epoch to identify whether the leader relationship has changed. Every time a leader is selected, it will have a new epoch, which identifies the current period of the leader's reign. The lower 32 bits are used for up counting.
epoch：可以理解为皇帝的年号，当新的皇帝leader产生后，将有一个新的epoch年号。
Each Server has three states during its work:
- LOOKING: The current server does not know who the leader is and is searching.
- LEADING: The current Server is the elected leader.
- FOLLOWING: The leader has been elected, and the current server is synchronized with it.
NO4: Why is Zookeeper designed like this?
The purpose of ZooKeeper's design is to provide high-performance, high-availability, and sequential-consistent distributed coordination services to ensure final data consistency.
高性能（简单的数据模型）
- Organize data nodes in a tree structure;
- All data nodes are stored in memory;
- Follower and Observer directly handle non-transactional requests;
高可用（构建集群）
- If more than half of the machines survive, the service can run normally
- Automatic leader election
顺序一致性（事务操作的顺序）
- Each transaction request will be forwarded to Leader for processing
- Each transaction will be assigned a globally unique incremental id (zxid, 64 bits: epoch + self-increment id)
最终一致性
- Guarantee the reliability of transaction submission through proposal voting
- The proposed voting method can only ensure that more than half of the nodes can see the latest data after the client receives the transaction submission successfully
NO5: Do you know what roles are in Zookeeper?
System model:
领导者（leader）
The Leader server provides read and write services for clients. Responsible for the initiation and resolution of voting, and update the system status.
学习者（learner）
followerFollower ( ) The Follower server provides read services for clients, participates in the Leader election process, and participates in the write operation "over half write success" strategy.- Observer (
observer) The Observer server provides read services for clients, does not participate in the Leader election process, and does not participate in the write operation "more than half of the write success" strategy. It is used to improve the read performance of the cluster without affecting the write performance.
客户端（client）: The originator of the service request.
NO6: Are you familiar with the Zookeeper node ZNode and related attributes?
What types of nodes are there?
Znode两种类型：
Persistent (persistent): After the client and server are disconnected, the created node is not deleted (default).
Ephemeral: After the client and server are disconnected, the created node is deleted by itself.
Znode有四种形式：
- Persistent directory node (PERSISTENT): After the client disconnects from Zookeeper, the node still has a persistent sequence number directory node (PERSISTENT_SEQUENTIAL)
- After the client disconnects from Zookeeper, the node still exists, but Zookeeper gives the node name sequentially number: Temporary Directory Node (EPHEMERAL)
- After the client disconnects from Zookeeper, the node is deleted: Temporary sequence numbering directory node (EPHEMERAL_SEQUENTIAL)
- After the client disconnects from Zookeeper, the node is deleted, but Zookeeper gives the node name a sequential number
"Note" : Set the sequence identifier when creating the ZNode. A value will be appended to the ZNode name. The sequence number is a monotonically increasing counter maintained by the parent node.
What are the node attributes
A znode node can not only store data, but also some other special attributes. Next, we create a /test node to analyze the meaning of its various attributes.
[zk: localhost:2181(CONNECTED) 6] get /test
456
cZxid = 0x59ac //
ctime = Mon Mar 30 15:20:08 CST 2020
mZxid = 0x59ad
mtime = Mon Mar 30 15:22:25 CST 2020
pZxid = 0x59ac
cversion = 0
dataVersion = 2
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 3
numChildren = 0
属性说明
NO7: Please briefly describe the main selection process of Zookeeper
Zookeeper的核心是原子广播，这个机制保证了各个Server之间的同步。实现这个机制的协议叫做Zab协议。Zab协议有两种模式，它们分别是恢复模式（选主）和广播模式（同步）。当服务启动或者在领导者崩溃后，Zab就进入了恢复模式，当领导者被选举出来，且大多数Server完成了和leader的状态同步以后，恢复模式就结束了。状态同步保证了leader和Server具有相同的系统状态。leader选举是保证分布式数据一致性的关键。
出现选举主要是两种场景：初始化、leader不可用。
当zk集群中的一台服务器出现以下两种情况之一时，就会开始leader选举。
（1）服务器初始化启动。
（2）服务器运行期间无法和leader保持连接。
而当一台机器进入leader选举流程时，当前集群也可能处于以下两种状态。
（1）集群中本来就已经存在一个leader。
（2）集群中确实不存在leader。
首先第一种情况，通常是集群中某一台机器启动比较晚，在它启动之前，集群已经正常工作，即已经存在一台leader服务器。当该机器试图去选举leader时，会被告知当前服务器的leader信息，它仅仅需要和leader机器建立连接，并进行状态同步即可。
重点是leader不可用了，此时的选主制度。
投票信息中包含两个最基本的信息。
sid：即server id，用来标识该机器在集群中的机器序号。
zxid：即zookeeper事务id号。
ZooKeeper状态的每一次改变, 都对应着一个递增的Transaction id,，该id称为zxid.，由于zxid的递增性质, 如果zxid1小于zxid2,，那么zxid1肯定先于zxid2发生。创建任意节点，或者更新任意节点的数据， 或者删除任意节点，都会导致Zookeeper状态发生改变，从而导致zxid的值增加。
以（sid，zxid）的形式来标识一次投票信息。
例如：如果当前服务器要推举sid为1，zxid为8的服务器成为leader，那么投票信息可以表示为（1，8）
集群中的每台机器发出自己的投票后，也会接受来自集群中其他机器的投票。每台机器都会根据一定的规则，来处理收到的其他机器的投票，以此来决定是否需要变更自己的投票。
规则如下：
（1）初始阶段，都会给自己投票。
（2）当接收到来自其他服务器的投票时，都需要将别人的投票和自己的投票进行pk，规则如下：
优先检查zxid。zxid比较大的服务器优先作为leader。如果zxid相同的话，就比较sid，sid比较大的服务器作为leader。
NO8：有了解过watch机制吗？
简单地说：client端会对某个znode 注册一个watcher事件，当该znode发生变化时，这些client会收到ZooKeeper的通知，然后client可以根据znode变化来做出业务上的改变等。
经典使用场景：zookeeper为dubbo提供服务的注册与发现，作为注册中心，但是大家有没有想过zookeeper为啥能够实现服务的注册与发现吗？
这就不得不说一下zookeeper的灵魂 Watcher（监听者）。
什么是watcher？
watcher 是zooKeeper中一个非常核心功能 ，客户端watcher 可以监控节点的数据变化以及它子节点的变化，一旦这些状态发生变化，zooKeeper服务端就会通知所有在这个节点上设置过watcher的客户端 ，从而每个客户端都很快感知，它所监听的节点状态发生变化，而做出对应的逻辑处理。
简单的介绍了一下watcher ，那么我们来分析一下，zookeeper是如何实现服务的注册与发现。zookeeper的服务注册与发现，主要应用的是zookeeper的znode节点数据模型和watcher机制，大致的流程如下：
- 服务注册：服务提供者（Provider）启动时，会向zookeeper服务端注册服务信息，也就是创建一个节点，例如：用户注册服务com.xxx.user.register，并在节点上存储服务的相关数据（如服务提供者的ip地址、端口等）。
- 服务发现：服务消费者（Consumer）启动时，根据自身配置的依赖服务信息，向zookeeper服务端获取注册的服务信息并设置watch监听，获取到注册的服务信息之后，将服务提供者的信息缓存在本地，并进行服务的调用。
- 服务通知：一旦服务提供者因某种原因宕机不再提供服务之后，客户端与zookeeper服务端断开连接，zookeeper服务端上服务提供者对应服务节点会被删除（例如：用户注册服务com.xxx.user.register），随后zookeeper服务端会异步向所有消费用户注册服务com.xxx.user.register，且设置了watch监听的服务消费者发出节点被删除的通知，消费者根据收到的通知拉取最新服务列表，更新本地缓存的服务列表。
上边的过程就是zookeeper可以实现服务注册与发现的大致原理。
watcher有哪些类型？
znode节点可以设置两类watch，一种是DataWatches，基于znode节点的数据变更从而触发 watch 事件，触发条件getData()、exists()、setData()、 create()。
另一种是Child Watches，基于znode的孩子节点发生变更触发的watch事件，触发条件 getChildren()、 create()。
而在调用 delete() 方法删除znode时，则会同时触发Data Watches和Child Watches，如果被删除的节点还有父节点，则父节点会触发一个Child Watches。
watcher有什么特性？
watch对节点的监听事件是一次性的！客户端在指定的节点设置了监听watch，一旦该节点数据发生变更通知一次客户端后，客户端对该节点的监听事件就失效了。
如果还要继续监听这个节点，就需要我们在客户端的监听回调中，再次对节点的监听watch事件设置为True。否则客户端只能接收到一次该节点的变更通知。
NO9：那你说说Zookeeper有哪些应用场景？
数据发布与订阅
发布与订阅即所谓的配置管理，顾名思义就是将数据发布到ZooKeeper节点上，供订阅者动态获取数据，实现配置信息的集中式管理和动态更新。例如全局的配置信息，地址列表等就非常适合使用。
数据发布/订阅的一个常见的场景是配置中心，发布者把数据发布到 ZooKeeper 的一个或一系列的节点上，供订阅者进行数据订阅，达到动态获取数据的目的。
配置信息一般有几个特点:
- 数据量小的KV
- 数据内容在运行时会发生动态变化
- 集群机器共享，配置一致
ZooKeeper 采用的是推拉结合的方式。
- 推: 服务端会推给注册了监控节点的客户端 Wathcer 事件通知
- 拉: 客户端获得通知后，然后主动到服务端拉取最新的数据
命名服务
作为分布式命名服务，命名服务是指通过指定的名字来获取资源或者服务的地址，利用ZooKeeper创建一个全局的路径，这个路径就可以作为一个名字，指向集群中的集群，提供的服务的地址，或者一个远程的对象等等。
统一命名服务的命名结构图如下所示：
1、在分布式环境下，经常需要对应用/服务进行统一命名，便于识别不同服务。
- 类似于域名与IP之间对应关系，IP不容易记住，而域名容易记住。
- 通过名称来获取资源或服务的地址，提供者等信息。
2、按照层次结构组织服务/应用名称。
- 可将服务名称以及地址信息写到ZooKeeper上，客户端通过ZooKeeper获取可用服务列表类。
配置管理
程序分布式的部署在不同的机器上，将程序的配置信息放在ZooKeeper的znode下，当有配置发生改变时，也就是znode发生变化时，可以通过改变zk中某个目录节点的内容，利用watch通知给各个客户端 从而更改配置。
ZooKeeper配置管理结构图如下所示：
1、分布式环境下，配置文件管理和同步是一个常见问题。
- 一个集群中，所有节点的配置信息是一致的，比如 Hadoop 集群。
- 对配置文件修改后，希望能够快速同步到各个节点上。
2、配置管理可交由ZooKeeper实现。
- 可将配置信息写入ZooKeeper上的一个Znode。
- 各个节点监听这个Znode。
- 一旦Znode中的数据被修改，ZooKeeper将通知各个节点。
集群管理
所谓集群管理就是：是否有机器退出和加入、选举master。
集群管理主要指集群监控和集群控制两个方面。前者侧重于集群运行时的状态的收集，后者则是对集群进行操作与控制。开发和运维中，面对集群，经常有如下需求:
- 希望知道集群中究竟有多少机器在工作
- 对集群中的每台机器的运行时状态进行数据收集
- 对集群中机器进行上下线的操作
集群管理结构图如下所示：
1、分布式环境中，实时掌握每个节点的状态是必要的，可根据节点实时状态做出一些调整。
2、可交由ZooKeeper实现。
- 可将节点信息写入ZooKeeper上的一个Znode。
- 监听这个Znode可获取它的实时状态变化。
3、典型应用
- Hbase中Master状态监控与选举。
利用ZooKeeper的强一致性，能够保证在分布式高并发情况下节点创建的全局唯一性，即：同时有多个客户端请求创建 /currentMaster 节点，最终一定只有一个客户端请求能够创建成功
分布式通知与协调
1、分布式环境中，经常存在一个服务需要知道它所管理的子服务的状态。
a）NameNode需知道各个Datanode的状态。
b）JobTracker需知道各个TaskTracker的状态。
2、心跳检测机制可通过ZooKeeper来实现。
3、信息推送可由ZooKeeper来实现，ZooKeeper相当于一个发布/订阅系统。
分布式锁
处于不同节点上不同的服务，它们可能需要顺序的访问一些资源，这里需要一把分布式的锁。
分布式锁具有以下特性：写锁、读锁、时序锁。
写锁：在zk上创建的一个临时的无编号的节点。由于是无序编号，在创建时不会自动编号，导致只能客户端有一个客户端得到锁，然后进行写入。
读锁：在zk上创建一个临时的有编号的节点，这样即使下次有客户端加入是同时创建相同的节点时，他也会自动编号，也可以获得锁对象，然后对其进行读取。
时序锁：在zk上创建的一个临时的有编号的节点根据编号的大小控制锁。
分布式队列
分布式队列分为两种：
1、当一个队列的成员都聚齐时，这个队列才可用，否则一直等待所有成员到达，这种是同步队列。
a）一个job由多个task组成，只有所有任务完成后，job才运行完成。
b）可为job创建一个/job目录，然后在该目录下，为每个完成的task创建一个临时的Znode，一旦临时节点数目达到task总数，则表明job运行完成。
2、队列按照FIFO方式进行入队和出队操作，例如实现生产者和消费者模型。
NO10：知道监听器的原理吗？
- 创建一个Main()线程。
- 在Main()线程中创建两个线程，一个负责网络连接通信（connect），一个负责监听（listener）。
- 通过connect线程将注册的监听事件发送给Zookeeper。
- 将注册的监听事件添加到Zookeeper的注册监听器列表中。
- Zookeeper监听到有数据或路径发生变化时，把这条消息发送给Listener线程。
- Listener线程内部调用process()方法。
NO11：为什么Zookeeper集群的数目，一般为奇数个？
首先需要明确zookeeper选举的规则：leader选举，要求可用节点数量 > 总节点数量/2。
比如：标记一个写是否成功是要在超过一半节点发送写请求成功时才认为有效。同样，Zookeeper选择领导者节点也是在超过一半节点同意时才有效。最后，Zookeeper是否正常是要根据是否超过一半的节点正常才算正常。这是基于CAP的一致性原理。
zookeeper有这样一个特性：集群中只要有过半的机器是正常工作的，那么整个集群对外就是可用的。
也就是说如果有2个zookeeper，那么只要有1个死了zookeeper就不能用了，因为1没有过半，所以2个zookeeper的死亡容忍度为0；
同理，要是有3个zookeeper，一个死了，还剩下2个正常的，过半了，所以3个zookeeper的容忍度为1；
同理：
2->0; Two zookeepers, up to 0 zookeepers can be unavailable.
3->1; Three zookeepers, up to 1 zookeeper can be unavailable.
4->1; Four zookeepers, up to 1 zookeeper can be unavailable.
5->2; Five zookeepers, up to 2 zookeepers can be unavailable.
6->2; Two zookeepers, up to 0 zookeepers can be unavailable.
....
You will find a rule that the tolerance of 2n and 2n-1 is the same, both are n-1, so in order to be more efficient, why add an unnecessary zookeeper.
Zookeeper's election strategy also requires more than half of the nodes to agree to be elected leader. If it is an even number of nodes, it may lead to the same number of votes.
to sum up
For many interviewers, the interview routine is basically this, from the background to the principle, to the architecture system, to the inherent characteristics of Zookeeper, and finally the interviewer is required to tell the actual application scenarios of Zookeeper.
"If you learn more of the same skill, you won't say a word of asking for help."
Push recommended reading reading
Master Mybatis dynamic mapping, I have worked hard on all the original 2020