ZooKeeper series (two) - Zookeeper stand-alone environment and to build a clustered environment

First, build a stand-alone environment

1.1 Download

Download the corresponding version Zookeeper, here I downloaded version 3.4.14. Official Download: https: //archive.apache.org/dist/zookeeper/

# wget https://archive.apache.org/dist/zookeeper/zookeeper-3.4.14/zookeeper-3.4.14.tar.gz

1.2 unzip

# tar -zxvf zookeeper-3.4.14.tar.gz

1.3 Configuration Environment Variables

# vim /etc/profile

Add environment variables:

export ZOOKEEPER_HOME=/usr/app/zookeeper-3.4.14
export PATH=$ZOOKEEPER_HOME/bin:$PATH

Making the environment variable configuration to take effect:

# source /etc/profile

1.4 modify the configuration

Installation directory into the conf/directory, copy and modify configuration samples:

# cp zoo_sample.cfg  zoo.cfg

Specified data storage directory and log file directory (the directory without prior creation, the program automatically creates), the revised full configuration is as follows:

# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
dataDir=/usr/local/zookeeper/data
dataLogDir=/usr/local/zookeeper/log
# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1

Configuration parameters:

tickTime : a base time calculating unit. Session timeout example: N * tickTime;

initLimit : a cluster, and allow connections from the node to the master node synchronizing connection initialization time, expressed in multiples tickTime;

syncLimit : a cluster, the master node and sent from the Master node between the message, the request and response time length (heartbeat mechanism);

dataDir : data storage location;

dataLogDir : log directory;

the clientPort : a port for connecting client, default 2181

1.5 start

Because already configured environment variables, you can directly use the following command to start:

zkServer.sh start

1.6 verification

JPS verify whether the process has started, appeared QuorumPeerMainrepresents a successful start.

[root@hadoop001 bin]# jps
3814 QuorumPeerMain

Second, build a clustered environment

To ensure high availability clusters, cluster nodes Zookeeper best is odd, there are at least three nodes, so here demonstrates build a cluster of three nodes. Here I use three hosts were set up, host names are hadoop001, hadoop002, hadoop003.

2.1 modify the configuration

Extracting a zookeeper installation package, modify the configuration file zoo.cfg, as follows. After using the scp command to install package distributed to three servers:

tickTime=2000
initLimit=10
syncLimit=5
dataDir=/usr/local/zookeeper-cluster/data/
dataLogDir=/usr/local/zookeeper-cluster/log/
clientPort=2181

# server.1 这个1是服务器的标识，可以是任意有效数字，标识这是第几个服务器节点，这个标识要写到dataDir目录下面myid文件里
# 指名集群间通讯端口和选举端口
server.1=hadoop001:2287:3387
server.2=hadoop002:2287:3387
server.3=hadoop003:2287:3387

2.2 identifies the node

Respectively three hosts dataDirnew directory myidfile, and write the corresponding node identifier. Zookeeper clusters by myidthe file identification cluster nodes and nodes to node communication via the communication ports and port configuration above elections, elect Leader node.

Create a storage directory:

# 三台主机均执行该命令
mkdir -vp  /usr/local/zookeeper-cluster/data/

Create and node identification is written to myidthe file:

# hadoop001主机
echo "1" > /usr/local/zookeeper-cluster/data/myid
# hadoop002主机
echo "2" > /usr/local/zookeeper-cluster/data/myid
# hadoop003主机
echo "3" > /usr/local/zookeeper-cluster/data/myid

Start Cluster 2.3

Respectively on the three hosts, execute the following command to start the service:

/usr/app/zookeeper-cluster/zookeeper/bin/zkServer.sh start

2.4 Cluster Verification

After starting to use zkServer.sh statusto view each cluster node status. As shown in Figure: Three node processes were started successfully, and hadoop002 as leader node, hadoop001 and hadoop003 as a follower node.

More big data series can be found GitHub open source project : Big Data Getting Started