Foreword:
This article will introduce building a Zookeepercluster to support kafkathe last Zookeeperversion that needs to be a registry. Of course, you can also use this cluster as Dubboa registry.
Tip: Kafka v2.6~v2.7.1 requires zookeeper v3.5.8+ support. Kafka v2.8 and above will not need Zookeeper as a registry. Click to view the official introduction of kafka
Versions used for this tutorial:
| name | Version |
|---|---|
| operating system | cents 7 |
| zookeeper | 3.5.9 |
| kafka (package) | 2.7.1 |
| JDK (optional) | 11.0.12 |
1. Download Zookeeper
1. How to choose the zookeeper installation package
- For online server use, select
stablethe latest version in the version.
Reason: The latest version often has bugs, and it is not recommended to use it online. - Select the format of the installation package
apache-zookeeper-{version}-bin.tar.gz, and the name must contain-bin
Reason: If you selectapache-zookeeper-{version}.tar.gzthe package, after decompression, you will find that there is nolibfolder in it, and an error will be reported when you start it, and the error code is:
Error: Could not find or load main class org.apache.zookeeper.server.quorum.QuorumPeerMain.
2. Download address:
| Version | address |
|---|---|
| Latest stable version | https://downloads.apache.org/zookeeper/stable/ |
| All versions | https://downloads.apache.org/zookeeper/ |
2. Cluster deployment
Goal: Build a zookeeper cluster with one master node and two slave nodes.
1. Prepare three linuxhosts
This article takes centorsas an example, the addresses of the three servers ipare:
10.39.88.58
10.39.88.59
10.39.88.60
2. Download the zookeeper compressed package
Enter the three server directories usr/localand execute:
cd /usr/local
wget https://downloads.apache.org/zookeeper/zookeeper-3.5.9/apache-zookeeper-3.5.9-bin.tar.gz
Note: With kafka v2.7.1 version, it is recommended to use apache-zookeeper-3.5.9-bin.tar.gz, do not use zookeeper3.6.*, an error will occur when kafka connects to zookeeper. Therefore, the zookeeper version is not as new as possible. You should download the corresponding zookeeper version according to the prompts on the kafka official website.
3. Unzip the compressed package
On all three servers, execute the following commands, unzip and rename the folder name to zookeeper(code cleanliness, folder name must be elegant):
tar -zxvf apache-zookeeper-3.5.9-bin.tar.gz
mv ./apache-zookeeper-3.5.9-bin zookeeper
4. Modify the configuration file
On the three servers, go into the configuration folder:
cd zookeeper/conf/
Modify the following configuration file, name it zoo.cfg, and copy it to the above folder (copy it completely, word for word!!!):
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
#dataDir=/tmp/zookeeper
# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1
dataDir=/usr/local/zookeeper/data/
server.1=10.39.88.58:2888:3888
server.2=10.39.88.59:2888:3888
server.3=10.39.88.60:2888:3888
Just modify the servercontent in the above file and change the ip to your three host ip addresses.
Example: If your three host IPs are 10.0.0.1, 10.0.0.2, 10.0.0.3, then modify them to:
server.1=10.0.0.3:2888:3888
server.2=10.0.0.2:2888:3888
server.3=10.0.0.3:2888:3888
Note: 2888:3888Do not change, its meaning zookeeper配置文件详解is explained at the end of the text.
Note: This configuration file is exactly the same for the three servers! ! !
5. Create myid file
What is myid file?
The myid file is used to distinguish the number of each host in the zookeeper cluster, and is a unique and non-repeatable numeric id.
myid content
myidThe file content format is required to be a number and must be equalserver.1=10.39.88.58:2888:3888to theserver.1number in the configuration file.
Example: write in the configuration file server.666=10.39.88.58:2888:3888, then this machine myidis 666.
Where is the myid file placed?
myidThe file should be stored indataDirthe path specified by the configuration file.
In the previous step, there is such a configuration in the configuration file: dataDir=/usr/local/zookeeper/data/, this address indicates that the zookeeper data is mounted here. We should put the myid file under this path.
Summarize:
myidThe file contents of the three services must be different from each other.- Each server's must be the same as in the
myidconfiguration file .sever.xx myidThe file is stored indataDirthe path specified by the configuration file.
6. Start the cluster
Enter the three linux hosts respectively, enter the zookeeper installation directory: usr/local/zookeeper/bin, execute the command:
./zkServer.sh start
7. Test whether the cluster is successfully started
Enter the three linux hosts respectively, enter the zookeeper installation directory: usr/local/zookeeper/bin, execute the command:
./zkServer.sh status
If the following information is displayed, it means that the host is elected as the leader (master node), that is, the startup is successful.
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost. Client SSL: false.
Mode: leader
If the following information is displayed, it means that the host is elected as a follower (slave node), that is, the startup is successful.
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost. Client SSL: false.
Mode: follower
At this point, the zookeeper cluster deployment has ended perfectly~
3. Detailed explanation of zookeeper configuration file
When we configured the cluster above, we took the meaning of the parameters of the configuration file in one stroke. Now I list the meanings below for your detailed review:
| Attributes | meaning |
|---|---|
| tickTime=2000 | It means that 1 tick means 2000ms in zookeeper, and how many ticks are used in all other places where time is used |
| initLimit = 10 | When initializing the connection between the follower server and the leader server in the cluster, the maximum number of heartbeats that can be tolerated is 10 ticks |
| syncLimit=5 | Indicates that the heartbeat time between the fowler and the leader is 5 ticks |
| clientPort=2181 | The port on which the client connects to the zookeeper service. This is a TCP port. |
| dataDir=/usr/local/zookeeper/data/ | data file directory |
| server.1=10.39.88.58:2888:3888 | List of services in the cluster. Port 2888 is used by zookeeper servers to communicate with each other; port 3888 is used for master node leader election. You can choose any port you like |
4. The operation and maintenance manual of the zookeeper cluster
Note: Enter the zookeeper installation directory binfolder and execute the command:
1. Common commands
# 启动：
./zkServer.sh start
# 关闭：
./zkServer.sh stop
# 查看运行状态，以及当前节点是leader还是follower
./zkServer.sh status
# 重启：
./zkServer.sh restart
2. View the running log
The directory where the running log of zookeeper is located:
/usr/local/zookeeper/logs
log files such as:zookeeper-root-server-i-6czujhq1.out
Summarize
This article builds a zookeeper cluster from scratch, explains the places that need to be configured, and the pits that are often encountered, and explains the meaning of each attribute in the configuration file, laying the foundation for the future deployment of kafka. If you like this tutorial, please like it for future use.