CentOS 7 Zookeeper introduction and cluster installations

Introduce the principle of

ZooKeeper is an open source distributed application coordination service, which includes a simple set of primitives, distributed applications based on its synchronization service, maintenance and configuration naming services.

Zookeeper designed

1. Eventual consistency: client whether to connect to the Server, it is presented to the same view.
2. Reliability: simple, robust, good performance, if the message m is received to a server, the message m will be received by all servers.
3. Real-time: Zookeeper ensure that clients will receive updated information server within a range of time intervals, or the information server failure. However, due to network delay and other reasons, Zookeeper not guarantee that two clients can simultaneously receive data just updated, the latest data if needed, should call sync before reading data () interface.
4. Waiting is not (wait-free): slow or failure of the client shall not interfere with the client's request quickly, so that each client can effectively wait.
5. Atomicity: Update only success or failure, no intermediate state.
6. Sequence: the partial order and includes a global ordered two types: global order means that if a message before the message is posted on a server b, a message will be posted on the message before all b Server; partial order means that if a message b Post same after a message sender, a row in front of bound b.

Zookeeper Works

In zookeeper cluster, each node there are the following three kinds of roles and four states:

• Role: leader, follower, observer
• 状态：leading,following,observing,looking

Zookeeper is the core of atomic broadcast, this mechanism ensures synchronization between the various Server. Protocol Implementation of this mechanism is called Zab protocol (ZooKeeper Atomic Broadcast protocol). Zab protocol has two modes, which are recovery mode (selected from the master Recovery) mode and broadcast (Broadcast synchronization). When the service starts or after leader crashes, Zab entered recovery mode, when the leader is elected, and most finished after Server synchronization and leader of the state, recovery mode is over. State synchronization ensures the leader and Server have the same system state.

In order to ensure consistency of the order of affairs, zookeeper using increasing transaction id number (zxid) to identify the transaction. All proposals (proposal) are proposed to be added when zxid. Implementation zxid is a 64-bit number that is high 32 epoch used to identify the relationship between the leader has changed every time a leader is elected, it will have a new epoch, the current logo belonging to the reign of the leader. 32 for the low counts.

Each Server There are four states in the course of their work:

• LOOKING: Current Server does not know who is the leader, we are searching for.
• LEADING: Current Server is the elected leader.
• FOLLOWING: leader has been elected, the current Server to synchronize.
• OBSERVING: observer act exactly the same in most cases with the follower, but they do not participate in the elections and vote, but only to accept the results (observing) elections and voting.

Zookeeper cluster nodes

• Zookeeper more nodes are deployed, the higher the reliability of service, the proposed deployment of an odd number of nodes, because zookeeper cluster is down more than half the number will make the entire cluster downtime.
• We need to give each zookeeper about 1G memory, if possible, preferably with a separate disk, because the independent disks zookeeper to ensure high performance. If your cluster load is heavy, do not zookeeper and RegionServer run on the same machine above, as DataNodes and TaskTrackers same.

Zookeeper cluster installation

Zookeeper need to run java environment, you need to install jdk

Note: Each server needs to be installed above zookeeper, jdk, recommended local download the installation package you need, and then uploaded to the server above, the above server download speed is too slow.

Environmental equipment

CPU name	system	IP addresses	ZK version	JDK version
node01	Centos7.5	172.16.1.11	3.4.13	1.8.0_192
node02	Centos7.5	172.16.1.12	3.4.13	1.8.0_192
node03	Centos7.5	172.16.1.13	3.4.13	1.8.0_192

JDK installation

We are required to do the following three machines are on

Download JDK's official website: http: //www.oracle.com/technetwork/java/javase/downloads/index.html

JDK Baidu network disk Download: https://pan.baidu.com/s/1FY90URiD6vEtnckR0kRSFQ extraction code: aiou

After downloading uploaded on three servers.

In the first step the following example, two behind the other servers need to operate this step;

cd /opt/soft/

unzip jdk1.8.0_192.zip

mv jdk1.8.0_192 /opt/

ln -s /opt/jdk1.8.0_192 /opt/jdk

ll /opt/jdk* -d
# lrwxrwxrwx 1 root root  17 Mar  1 00:02 /opt/jdk -> /opt/jdk1.8.0_192
# drwxr-xr-x 7 root root 245 Oct  6  2018 /opt/jdk1.8.0_192

Configure the environment variables, where only a single configuration for the root user, not arranged global environment variables;

cat >> /root/.bash_profile<<EOF
export JAVA_HOME=/opt/jdk
export JRE_HOME=\$JAVA_HOME/jre
export PATH=\$PATH:\$JAVA_HOME/bin
export CLASSPATH=./:\$JAVA_HOME/lib:\$JAVA_HOME/jre/lib
EOF

source /root/.bash_profile 

java -version

# java version "1.8.0_192"
# Java(TM) SE Runtime Environment (build 1.8.0_192-b12)
# Java HotSpot(TM) 64-Bit Server VM (build 25.192-b12, mixed mode)

zookeeper installation

• Baidu network disk link: https: //pan.baidu.com/s/1QP9RLVj0ikKwhPf98rYjOA extraction code: dmw6
• Zookeeper download link address: https: //archive.apache.org/dist/zookeeper/
• Official website: http: //zookeeper.apache.org/

We are required to do the following three machines are on

# 需要添加所有机器自身的主机名解析
echo >> /etc/hosts <EOF
172.16.1.11 node01
172.16.1.12 node02
172.16.1.13 node03
EOF

cd /opt/soft/

tar xf zookeeper-3.4.13.tar.gz 

mv zookeeper-3.4.13 /opt/

ln -s /opt/zookeeper-3.4.13 /opt/zookeeper_cluster

ls -ld /opt/zookeeper*

# drwxr-xr-x 10  501 games 4096 Jul  1  2018 /opt/zookeeper-3.4.13
# lrwxrwxrwx  1 root root    21 Mar  1 00:50 /opt/zookeeper_cluster -> /opt/zookeeper-3.4.13

Configuration

node01 node configuration

mkdir /opt/zookeeper_cluster/{data,logs}

echo "1" > /opt/zookeeper_cluster/data/myid

cd /opt/zookeeper_cluster/conf/

cat > zoo.cfg <<EOF
tickTime=2000
initLimit=10
syncLimit=5
dataLogDir=/opt/zookeeper_cluster/logs
dataDir=/opt/zookeeper_cluster/data
clientPort=2181
autopurge.snapRetainCount=500
autopurge.purgeInterval=24
server.1=172.16.1.11:2888:3888
server.2=172.16.1.12:2888:3888
server.3=172.16.1.13:2888:3888
EOF

node02 node configuration

mkdir /opt/zookeeper_cluster/{data,logs}

echo "2" > /opt/zookeeper_cluster/data/myid

cd /opt/zookeeper_cluster/conf/

cat > zoo.cfg <<EOF
tickTime=2000
initLimit=10
syncLimit=5
dataLogDir=/opt/zookeeper_cluster/logs
dataDir=/opt/zookeeper_cluster/data
clientPort=2181
autopurge.snapRetainCount=500
autopurge.purgeInterval=24
server.1=172.16.1.11:2888:3888
server.2=172.16.1.12:2888:3888
server.3=172.16.1.13:2888:3888
EOF

node03 node configuration

mkdir /opt/zookeeper_cluster/{data,logs}

echo "3" > /opt/zookeeper_cluster/data/myid

cd /opt/zookeeper_cluster/conf/

cat > zoo.cfg <<EOF
tickTime=2000
initLimit=10
syncLimit=5
dataLogDir=/opt/zookeeper_cluster/logs
dataDir=/opt/zookeeper_cluster/data
clientPort=2181
autopurge.snapRetainCount=500
autopurge.purgeInterval=24
server.1=172.16.1.11:2888:3888
server.2=172.16.1.12:2888:3888
server.3=172.16.1.13:2888:3888
EOF

Up and check the service status

# 三个节点都执行
/opt/zookeeper_cluster/bin/zkServer.sh start

node01

[root@node01 bin]# /opt/zookeeper_cluster/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /opt/zookeeper_cluster/bin/../conf/zoo.cfg
Mode: follower

node02

[root@node02 bin]#/opt/zookeeper_cluster/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /opt/zookeeper_cluster/bin/../conf/zoo.cfg
Mode: follower

node03

[root@node03 bin]# /opt/zookeeper_cluster/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /opt/zookeeper_cluster/bin/../conf/zoo.cfg
Mode: leader