Chapter One: Zookeeper Introduction
Zookeeper, animal administrator, is used to manage hadoop (elephant), Hive (Bee), Pig (pig) administrator.
Apache Solr and Apache Hbase distributed clusters have used the Zookeeper.
Zookeeper is a distributed, open the program coordination services , is a subproject of Hadoop project. The main functions it provides configuration management , name services , distributed lock , cluster management .
1.1: The role of Zookeeper: configuration management (focus)
Throws question:
In our application, in addition to the code, there are some that various configurations. Such as: database connectivity. Generally, we are using a form of the profile, the introduction of these profiles in your code.
When we only have one configuration, only one server, and do not often make changes when using a configuration file is a good practice. However, if our configuration is very large, and many servers need this, then, using a configuration file request is not a good choice.
This time we need to find a way to centrally manage the configuration, we modify the configuration of this central place, all interested can obtain the configuration file changes, Zookeeper on such a service.
problem solved:
Zookeeper use Zab such coherence protocol to provide consistency. Now a lot of open source projects use Zookeeper to maintain your configuration.
Such as HBase, the client is connected to a Zookeeper, HBase cluster configuration to obtain the necessary information before you can carry out further operations.
Open source message queue Kafka, a Zookeeper also use to maintain the information broker.
In the open source SOA framework Dubbo Alibaba is also widely used Zookeeper some configuration management to implement the service management
1.2: The role Zookeeper: Name Service (understand)
Question Throws:
For example, to access the system via a network, we have to know each other's IP address, but IP addresses is very complex, this time you need to use to access work domain name. However, the computer is not a domain name, there is a solution that we have a domain name to IP mappings for you to back up each machine, this can solve part of the problem, but how if the domain name corresponding to the IP changes do? So there DNS, we only need to access one point we are all familiar (known), and it will tell you what the domain name corresponding IP Yes.
problem solved:
This problem also exists in our applications, especially in a particularly large number of our services, if we save the address of the service in the local time will be very inconvenient, but if we only need to access one we are all familiar access points to provide here a unified entrance , it will be very easy to maintain
1.3: The role of Zookeeper: Distributed Lock (understand)
Question Throws:
For example: In a distributed environment, in order to improve the reliability of the cluster each server we deploy the same service, but one thing if the cluster each server were, then you need to be each other coordination , programming it will be very complicated. And if we just let a service operation, and that there is a single point of issue .
problem solved:
The use of distributed lock , at some point so that only a service to go to work, that this time the lock release service problems, immediately fail over to another service. In many distributed systems, they are doing so, also known as Leader Election .
For example: HBase The Master is using this mechanism, it should be noted that the same process distributed lock and lock or differentiated, so when in use, the same process than the more cautious use of the lock.
1.4: The role of Zookeeper: cluster management (focus)
Question Throws:
In a distributed cluster, often due to various reasons, such as hardware failures, software failures, network problems, some nodes will be coming and going. There are new node to join in, but also the old node from the cluster .
This time, the other machines in the cluster need to perceive this change , and then make the appropriate response in accordance with this change.
problem solved:
For example, we are a distributed storage system, there is a central allocation is responsible for storing the control node, we need to allocate storage node cluster according to the state now when a new store coming in. Then we need to dynamically sense the current state of the cluster .
For example, a distributed SOA architecture, the service is provided by a cluster, when consumers visit a service, you need to use some kind of mechanism to find which nodes can now provide this service (also known as service discovery such as open source SOA framework Dubbo Alibaba on the use of the underlying mechanism of service discovery as a Zookeeper). There are open-source Kafka queue on the use of upper and lower Zookeeper as Cosnumer management.
Chapter II: Zookeeper storage structure
2.1: Znode
In the Zookeeper, znode is a Unix file system path similar to the root node, the node to be stored or acquired data .
Zookeeper is a set of underlying data structures. This data structure is a tree structure , the above each node, called " znode ", Zookeeper data is stored according to a tree structure, but znode node further divided into four different types.
Each node znode default store data of 1MB ( only used to record state ).
You can use zkCli command, log on to the Zookeeper, and through ls, create, delete, get, set to operate these znode node commands.
2.2: Znode node type
(1 ) PERSISTENT (persistent node) : The so-called persistent node refers to the node is created, it will persist until the delete operation to remove the active node, otherwise it will not create the node failure of client session disappear.
(2) PERSISTENT_SEQUENTIAL (persistent node order) : the basic characteristics of such nodes, and nodes are the same persistence. Additional features are, in ZK, each parent will maintain a first-class timing for its child nodes, the order will be recorded each child node created. Based on this characteristic, at the same time create a child node, you can set this property, then the process of creating a node, ZK will be automatically given node plus a numeric suffix, as the new node name. The numeric suffix range is the maximum range of integer values. While creating a node, only we need to pass the node "/ test_", so, Zookeeper will automatically give back to add numbers.
(3) Ephemeral (temporary node) : and persistent node is different, and the life cycle of a client node session temporary binding. In other words, if the client session fails, then the node will automatically be removed. Note that, here mentioned session failover, rather than disconnected. In addition, you can not create a child node in a temporary node.
It is noteworthy that, when the client fails, the node is not generated suddenly disappear, but also over a period of time, probably within 10s.
(4) EPHEMERAL_SEQUENTIAL (temporary automatic numbering node) : This node node are temporary, but with the order, the client session ends, the node disappears.
Chapter III: Zookeeper installation (single version)
Installation Environment: Linux system, JDK
installation steps:
The first step: Upload Zookeeper archive
Resource Pack can be downloaded from the official site zookeeper.apache.com. The latest release version: 3.4.12.
Step two: extracting archive Zookeeper
tar -zxf zookeeper-3.4.6.tar.gz
cp zookeeper-3.4.6 /usr/local/zookeeper -r
Step Three: Configure Zookeeper
Zookeeper at boot time, the default conf directory to find the next zoo.cfg a configuration file named.
cp zoo_sample.cfg zoo.cfg
Zoo.cfg modify the configuration file:
Set dataDir path: the path to the data cache, you need to create a data folder
Step Four: Start Zookeeper
In the / bin directory, start zkServer, sh file
(1) Load the default configuration file: ./ zkServer.sh start, will go to the default configuration file conf directory to load zoo.cfg
(2) loading configuration file specifies: ./ zkServer.sh start [profile path position]
Stop Zookeeper: ./ zkServer.sh stop
View Zookeeper status: ./ zkServer.sh status
zookeeper file directory:
Chapter IV: Zookeeper cluster principle
4.1: Zookeeper cluster roles
Zookeeper role in the cluster mainly in the following three categories:
4.2: designed
(1) The final consistency: Client connection regardless of where a server, are shown with a view, this is the most important performance zookeeper.
(2) Reliability: If the message m is received by a server, then it will be received by all servers
（3）实时性：Zookeeper保证客户端将在一个时间间隔范围内，获得服务器的更新信息，或者服务器失效的信息；但由于网络延迟的原因，zookeeper不能保证两个客户端能同时得到刚更新的数据，如果需要最新的数据，应该在读数据之前调用sync（）接口
（4）等待无关（wait-free）：慢的或者实效的client不得干预快速的client请求，使得每个client都能有效的等待
（5）原子性：更新只有成功或者失败，没有中间状态
（6）顺序性：包括全局有序和偏序两种：全局有序是指如果一台服务器上消息a在消息b之前发布，则在所有的server上消息a都会在消息b之前发布；偏序是指如果一个消息b在消息a后被同一个发布者发布，消息a必将排在消息b之前。
第五章：Zookeeper创建集群
使用三个zookeeper搭建一个伪集群，应用部署在linux系统：192.168.110.128；服务器监听端口为2181、2182、2183；投票选举端口为2881/3881、2882/3882、2883/3883。
三个集群都在：/usr/local/zookeepercluster目录下
子目录分别为：zookeeper01、zookeeper02、zookeeper03
安装方法与单机版一样。配置文件需要指定data目录。
特殊地方：
每个集群需要提供应用唯一标识：
在zookeeper集群中，每个节点需要一个唯一标识，这个唯一标识要求必须是一个自然数，且唯一标识保存位置是：/data/myid，其中myid为配置文件。
配置方案：
（1）手动用vi命令编辑myid，并添加标识1，2，3
（2）使用简化命令：echo [唯一标识] >> myid
echo命令为回声命令，系统会将命令发送数据返回，“>>”为定位，代表系统回声数据指定发送到什么位置，此命令代表系统回声数据发送到myid文件中，如果没有文件则创建文件。
通常需要给配置文件添加客户自行权限，命令如下：
chmod +x 文件名
修改配置文件zoo.cfg:设置服务、投票、选举端口
vi zoo.cfg（每个集群都需要配置）
clientPort=2181 #服务端口根据应用做对应修改,zk01-2181,zk02-2182,zk03-2183 server.1=192.168.70.143:2881:3881 server.2=192.168.70.143:2882:3882 server.3=192.168.70.143:2883:3883
启动zookeeper应用:
在bin/zkServer.sh.start
注：zookeeper集群搭建完成后，至少需要启动两个应用才能提供服务，因为需要选举出主服务节点。启动所有的zookeeper节点后，可以使用命令：bin/zkServer.sh status来查看节点状态，如下所示：
Model:leader 表示主机
Model：follower 表示备份机
关闭zookeeper应用：
命令：bin/zkServer.sh stop
第六章：Zookeeper常见命令
启动zookeeper客户端：
bin目录下：./zkCli.sh
（1）当输入命令不正确时，会提示信息：
（2）常用命令：ls
使用格式：ls path
表示列表路径下的资源
指的注意的是：在zookeeper中，
根节点路径： ls /
子节点路径： ls /zookeeper
下一级子节点：ls /zookeeper/quota
（3）常用命令：quit
表示：退出客户端
（4）连接指定的ip地址下的服务器
./zkCli.sh -server 192.168.110.128:2182
（5）常用命令：connect 【ip】
表示：连接到指定的服务器
例如：connect 192.168.110.128:2183
（6）常用命令：create 【-e】【-s】 path data
表示：创建节点
-e：表示创建临时节点，可以不指定
-s：表示创建一个顺序节点，可以不指定
path：表示在哪个节点下创建新的节点，其中，根节点表示为/
data：表示节点数据
例如：create /test 123
表示：在根节点下，创建一个test节点，数据为123，节点类型没有给定，默认为持久型节点。
例如:创建顺序节点：create /test2 2
zookeeper自动为节点添加后缀信息；
（7）常用命令：get path
表示：查看指定节点的数据
例如：get /test
123 cZxid = 0x400000004 ctime = Fri Jul 05 18:00:23 CST 2019 mZxid = 0x400000004 mtime = Fri Jul 05 18:00:23 CST 2019 pZxid = 0x400000004 cversion = 0 dataVersion = 0 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 3 numChildren = 0
（8）常用命令：set path data 【version】
表示：设置对应位置节点的数据
例如：
（9）delete path 【version】
表示：删除指定节点，此命令不能删除有子节点的节点
（10）常用命令：rmr path
表示：删除指定节点，包括子节点
（11）常用命令：quit
表示：退出控制台