Zookeeper cluster construction (required for big data platforms)

foreword

Zookeeper is one of the necessary services for big data platforms. Zookeeper management is required in Hadoop clusters, Kafka clusters, and Hbase clusters, for example.

Preparation

1. Three Linux hosts need to be prepared, which can be Centos, Ubuntu, etc.
2. Configure Java environment variables on the three Linux hosts.
3. Configure SSH password-free login for the three hosts.

start building

1. Download the Zookeeper installation package, here is version 3.1.14, just use wget to download it.

wget https://mirrors.tuna.tsinghua.edu.cn/apache/zookeeper/zookeeper-3.4.14/zookeeper-3.4.14.tar.gz

2. After downloading, use the tar command to decompress the installation package:

tar -zxvf zookeeper-3.4.14.tar.gz

3. Write a configuration file

# 切换到Zookeeper的conf目录下
cd zookeeper-3.4.14/conf
#  拷贝一份配置
cp zoo_sample.cfg zoo.cfg
# 使用vim打开配置文件进行编辑
vim zoo.cfg

zoo.cfg

tickTime=2000
initLimit=10
syncLimit=5
# 数据存储目录
dataDir=/tmp/zookeeper
# Zookeeper的端口
clientPort=2181
# 集群节点的配置
# 其中server.0的0为Zookeeper集群中的节点id
# 2888为原子广播端口，3888为选举端口
server.0=192.168.0.12:2888:3888
server.1=192.168.0.11:2888:3888
server.2=192.168.0.6:2888:3888

4. Write the myid configuration file.
   The content in the myid configuration file corresponds to the server.id configured in zoo.cfg. For example, the server.0 node needs to configure a "0" in the myid configuration file, and so on.
   The location of the myid configuration file is the directory of dataDir in zoo.cfg.
   Write the configuration content to the myid file through redirection.

echo 0 > /tmp/zookeeper/myid

5. Remotely copy the decompressed and configured Zookeeper directory to the other two machines

scp -r zookeeper-3.4.14 [email protected]:/home/worker/bigdata/

The configuration does not need to be modified, just configure the corresponding myid file.

echo 2 > /tmp/zookeeper/myid

And so on for the third machine.

start zookeeper

After all three machines are configured, start Zookeeper for testing. Since no environment variables are configured, you need to enter the Zookeeper bin directory to execute commands.
Start Zookeeper with the command

./zkServer.sh start

Execute the same command on the other two machines
and then use the following command to check whether the startup is successful:

./zkServer.sh status

If the status is displayed in Mode, the configuration is successful. Two followers and one leader.
We can also use zkClinet for testing, and also use the following Minglin to log in to the Zookeeper cluster in the bin directory

./zkCli.sh

Login success screen:

A directory with zookeeper at the root node is successful.
If these are not displayed, there may be some problems. There is also a zookeeper.out file in the bin directory, which stores the running logs of Zookeeper. You can check the logs inside to determine what is wrong. .