zookeeper download and install the electoral mechanism (a zookeeper)

1. Brief Overview

Zookeeper is an open source distributed, to provide coordinated services for distributed application framework.
Zookeeper mode from the design point of view to understand: is a service-based distributed management framework Observer pattern design which is responsible for data storage and management of concern to everyone, and then accept the registration of the observer, but a change in the status of these data, it Zookeeper He will be responsible for notifying those observers have been registered on the Zookeeper react accordingly, in order to achieve a similar cluster Master / Slave management
Zookeeper = + file system notification mechanism

2. Features

1. Zookeeper: a leader (leader), more followers (follower) consisting of a cluster
2. Leader is responsible for initiating and vote on the resolution, updates the system status
3. Follower for receiving a client request results returned to the client, to vote in the election process Leader
   4) as long as more than half of the cluster nodes survival, Zookeeper cluster will be able to normal service.
   Consistent 5) Global data: Each serer retain an identical copy of the data, client whether it is connected to which serer, the data are consistent.
   6) order the update request, the update request from the same client to execute in the order they were sent
   7) atomic data updating, a data update either succeed or fail.
   8) real-time, within a certain time frame, client can read the latest data.

3. Data structure

Zookeeper data model knot Jin and Unix file systems are very similar, on the whole can be seen as a bare tree, each node is called a Znode, it is clear that zookeeper cluster itself safeguarding a set of data structures · The storage structure is a tree structure each node on which we call "znode", each capable of storing a default znode IMB data path can znode each uniquely identified by its

4. scenarios

Services provided include: distributed messaging synchronization and coordination mechanisms on the server node offline dynamic, unified configuration management load balancing, cluster management
under 1) a distributed environment, configuration file management and synchronization is a common problem
in a cluster, all nodes configuration think the letter is the same, such as Hadoop cluster.
After modifying the configuration file, hoping to quickly synchronize to each node.
2) Configuration management can be handed over to Zookeeper realized.
Configuration information can be written on a znode ZooKeeper.
Each node listens to this Znode.
Once the data is modified Znode, Zookeeper notifies each node.

 

image.png

 

There are a lot of scenes follow-up study to add it again

5. Download and install

Open the following address
http://zookeeper.apache.org/releases.html
click on download

image.png

 

5.1 Select an address download

image.png

Click to choose a version 5.2 you will see tar.gz

image.png

5.3 uploaded to our 102 machine

image.png

5.4 can be seen successfully uploaded

[shaozhiqi@hadoop102 ~]$ cd /opt/software/
[shaozhiqi@hadoop102 software]$ ll
total 776464
-rw-rw-r--. 1 shaozhiqi shaozhiqi 332433589 Jun 23 19:59 hadoop-3.1.2.tar.gz -rw-rw-r--. 1 shaozhiqi shaozhiqi 194990602 Jun 23 19:59 jdk-8u211-linux-x64.tar.gz -rw-rw-r--. 1 shaozhiqi shaozhiqi 229988313 Jun 30 17:46 spark-2.4.3-bin-hadoop2.7.tgz -rw-rw-r--. 1 shaozhiqi shaozhiqi 37676320 Jul 2 21:50 zookeeper-3.4.14.tar.gz [shaozhiqi@hadoop102 software]$ 

5.5 unzip jar

[shaozhiqi@hadoop102 software]$ tar -zxvf zookeeper-3.4.14.tar.gz -C /opt/module/

Enter the directory to modify configuration

[shaozhiqi@hadoop102 zookeeper-3.4.14]$ ll
total 1684
drwxr-xr-x. 2 shaozhiqi shaozhiqi 4096 Mar 7 01:09 bin //我们常用的命令 -rw-rw-r--. 1 shaozhiqi shaozhiqi 97426 Mar 7 00:50 build.xml drwxr-xr-x. 2 shaozhiqi shaozhiqi 74 Mar 7 01:09 conf //我们要配置配置文件 drwxr-xr-x. 2 shaozhiqi shaozhiqi 4096 Mar 7 01:10 dist-maven -rw-rw-r--. 1 shaozhiqi shaozhiqi 1709 Mar 7 00:50 ivysettings.xml -rw-rw-r--. 1 shaozhiqi shaozhiqi 10742 Mar 7 00:50 ivy.xml drwxr-xr-x. 4 shaozhiqi shaozhiqi 4096 Mar 7 01:09 lib //依赖的jar -rw-rw-r--. 1 shaozhiqi shaozhiqi 11970 Mar 7 00:50 LICENSE.txt -rw-rw-r--. 1 shaozhiqi shaozhiqi 3132 Mar 7 00:50 NOTICE.txt -rw-rw-r--. 1 shaozhiqi shaozhiqi 31622 Mar 7 00:50 pom.xml -rw-rw-r--. 1 shaozhiqi shaozhiqi 1765 Mar 7 00:50 README.md -rw-rw-r--. 1 shaozhiqi shaozhiqi 1770 Mar 7 00:50 README_packaging.txt drwxr-xr-x. 3 shaozhiqi shaozhiqi 21 Mar 7 00:50 src -rw-rw-r--. 1 shaozhiqi shaozhiqi 1515359 Mar 7 00:50 zookeeper-3.4.14.jar -rw-rw-r--. 1 shaozhiqi shaozhiqi 836 Mar 7 01:10 zookeeper-3.4.14.jar.asc -rw-rw-r--. 1 shaozhiqi shaozhiqi 33 Mar 7 00:50 zookeeper-3.4.14.jar.md5 -rw-rw-r--. 1 shaozhiqi shaozhiqi 41 Mar 7 00:50 zookeeper-3.4.14.jar.sha1 drwxr-xr-x. 3 shaozhiqi shaozhiqi 45 Mar 7 01:09 zookeeper-client drwxr-xr-x. 12 shaozhiqi shaozhiqi 4096 Mar 7 01:09 zookeeper-contrib drwxr-xr-x. 7 shaozhiqi shaozhiqi 4096 Mar 7 01:09 zookeeper-docs drwxr-xr-x. 3 shaozhiqi shaozhiqi 33 Mar 7 01:09 zookeeper-it drwxr-xr-x. 4 shaozhiqi shaozhiqi 43 Mar 7 01:09 zookeeper-jute drwxr-xr-x. 5 shaozhiqi shaozhiqi 4096 Mar 7 01:09 zookeeper-recipes drwxr-xr-x. 3 shaozhiqi shaozhiqi 30 Mar 7 01:09 zookeeper-server [shaozhiqi@hadoop102 zookeeper-3.4.14]$ 

5.6 renamed zoo_sample.cfg

[shaozhiqi@hadoop102 conf]$ mv zoo_sample.cfg zoo.cfg

Modify the file storage address to our address dataDir = / opt / module / zookeeper -3.4.14 / zkdata
other to not change

[shaozhiqi@hadoop102 conf]$ vim zoo.cfg
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10 # The number of ticks that can pass between # sending a request and getting an acknowledgement syncLimit=5 # the directory where the snapshot is stored. # do not use /tmp for storage, /tmp here is just # example sakes. dataDir=/opt/module/zookeeper-3.4.14/zkdata # the port at which the clients will connect clientPort=2181 # the maximum number of client connections. 

6. Start a single node

6.1 start the server:

[shaozhiqi@hadoop102 zookeeper-3.4.14]$ cd bin/
[shaozhiqi@hadoop102 bin]$ pwd
/opt/module/zookeeper-3.4.14/bin [shaozhiqi@hadoop102 bin]$ ll total 44 -rwxr-xr-x. 1 shaozhiqi shaozhiqi 232 Mar 7 00:50 README.txt -rwxr-xr-x. 1 shaozhiqi shaozhiqi 1937 Mar 7 00:50 zkCleanup.sh -rwxr-xr-x. 1 shaozhiqi shaozhiqi 1056 Mar 7 00:50 zkCli.cmd -rwxr-xr-x. 1 shaozhiqi shaozhiqi 1534 Mar 7 00:50 zkCli.sh -rwxr-xr-x. 1 shaozhiqi shaozhiqi 1759 Mar 7 00:50 zkEnv.cmd -rwxr-xr-x. 1 shaozhiqi shaozhiqi 2919 Mar 7 00:50 zkEnv.sh -rwxr-xr-x. 1 shaozhiqi shaozhiqi 1089 Mar 7 00:50 zkServer.cmd -rwxr-xr-x. 1 shaozhiqi shaozhiqi 6773 Mar 7 00:50 zkServer.sh -rwxr-xr-x. 1 shaozhiqi shaozhiqi 996 Mar 7 00:50 zkTxnLogToolkit.cmd -rwxr-xr-x. 1 shaozhiqi shaozhiqi 1385 Mar 7 00:50 zkTxnLogToolkit.sh [shaozhiqi@hadoop102 bin]$ ./zkServer.sh start ZooKeeper JMX enabled by default Using config: /opt/module/zookeeper-3.4.14/bin/../conf/zoo.cfg Starting zookeeper ... STARTED [shaozhiqi@hadoop102 bin]$ 

Verify that started successfully: QuorumPeerMain

[shaozhiqi@hadoop102 bin]$ jps
20067 Jps
20036 QuorumPeerMain
[shaozhiqi@hadoop102 bin]$

6.2 Starting the client, you can see the log links to the 2181

[shaozhiqi@hadoop102 bin]$ ./zkCli.sh
Connecting to localhost:2181
2019-07-02 08:00:02,854 [myid:] - INFO [main:Environment@100] - Client environment:zookeeper.version=3.4.14-4c25d480e66aadd371de8bd2fd8da255ac140bcf, built on 03/06/2019 16:18 GMT 

Single node can start to prove our zookeeper is easy to use

7. Start Cluster

Stopped zookeeper server:

shaozhiqi@hadoop102 bin]$ ./zkServer.sh stop

7.1 Amendment zoo.cfg profile

Adding configure, 2888 on behalf of the leader port 3888 is hung up when the 2888, 3888 replaced
myid inside the id in the trunked mode designated 2,3,4, determined according to the following configuration for the server is

[shaozhiqi@hadoop102 conf]$ vim zoo.cfg
server.2=hadoop102:2888:3888 server.3=hadoop103:2888:3888 server.4=hadoop104:2888:3888 

7.2 Configuration of myid 102

[shaozhiqi@hadoop102 zookeeper-3.4.14]$ cd zkdata/
[shaozhiqi@hadoop102 zkdata]$ ll
total 0
drwxrwxr-x. 2 shaozhiqi shaozhiqi 18 Jul 2 23:00 version-2 [shaozhiqi@hadoop102 zkdata]$ vim myid [shaozhiqi@hadoop102 zkdata]$ cat myid 2 [shaozhiqi@hadoop102 zkdata]$ 

Our zookeeper 7.3 synchronized to our machine 103104

[shaozhiqi@hadoop102 module]$ testxsync zookeeper-3.4.14/

Modify myid 103 and 104

[shaozhiqi@hadoop103 zookeeper-3.4.14]$ cd zkdata/
[shaozhiqi@hadoop103 zkdata]$ vim myid
[shaozhiqi@hadoop103 zkdata]$ cat myid
3
[shaozhiqi@hadoop103 zkdata]$

[shaozhiqi@hadoop104 zkdata]$ vim myid
[shaozhiqi@hadoop104 zkdata]$ cat myid
4
[shaozhiqi@hadoop104 zkdata]$

Start Cluster 7.4

7.4.1 Zookeeper election mechanism

So we have three machines are configured, how it started when he was a leader election?
1) Half mechanism (Paxos protocol): More than half of the cluster machines to survive, the cluster is available. Therefore zookeeper adapted to fit in the odd bit machine
while 2) Zookeeper not specified in the configuration file of the master and slave. However, there is a zookeeper work node leader. Others are follower. Leader is elected interim mechanism within the generated
3) with a simple example to illustrate the entire electoral process ,, suppose you have five servers in the zookeeper clusters, their d from 1-5, and they are the latest to start is no historical data, at this point the amount of stored data, are the same, assuming that these servers sequential startup to see what happens.
leader produces Description
(1) servers 1 start, only this time it started a single server, it sends the message to no response, so it's been a state election LOOKING state.
(2) 2 server starts, it communicates with the server most started 1, to exchange their election results due neither historical data, the value of d larger servers to 2 win, but because there is no more than than half of it agreed to elect servers (more than half of this example is 3, so the server 1,2 or continue to maintain LOOKING iron status.
(3) server 3 starts, based on the above theoretical analysis, the server 3 becomes the server, 2, 3 the boss and the above difference is that this time there are three servers elected it, so it became the leader of the election,
(4) server 4 starts, the foregoing analysis, in theory, should be the server server 4 1 3,4 largest, but because there are already more than half of the elected server 3 server, so it can only be received when the life of the brother
(5) server 5 starts, as if with a little brother
According to this theory of our three The machine should be 103

7.4.2 Start 102

[shaozhiqi@hadoop102 bin]$ ./zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /opt/module/zookeeper-3.4.14/bin/../conf/zoo.cfg Starting zookeeper ... STARTED [shaozhiqi@hadoop102 bin]$ 

When the service can provide value from a service view status, error, more than half the number of service

[shaozhiqi@hadoop102 bin]$ ./zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /opt/module/zookeeper-3.4.14/bin/../conf/zoo.cfg Error contacting service. It is probably not running. [shaozhiqi@hadoop102 bin]$ 

7.4.3 Start 103, 103 are found to check the status of leader

[shaozhiqi@hadoop103 bin]$ ./zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /opt/module/zookeeper-3.4.14/bin/../conf/zoo.cfg Starting zookeeper ... STARTED [shaozhiqi@hadoop103 bin]$ ./zkServer.sh status ZooKeeper JMX enabled by default Using config: /opt/module/zookeeper-3.4.14/bin/../conf/zoo.cfg Mode: leader [shaozhiqi@hadoop103 bin]$ 

7.4.4 Start 104, view the status of follower 104, so 103 is the leader

[shaozhiqi@hadoop104 zookeeper-3.4.14]$ bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /opt/module/zookeeper-3.4.14/bin/../conf/zoo.cfg Starting zookeeper ... STARTED [shaozhiqi@hadoop104 zookeeper-3.4.14]$ bin/zkServer.sh status ZooKeeper JMX enabled by default Using config: /opt/module/zookeeper-3.4.14/bin/../conf/zoo.cfg Mode: follower [shaozhiqi@hadoop104 zookeeper-3.4.14]$ 

Continuously updated later in ~~~