Configuring Apache Zookeeper Distributed Cluster on Linux Multi-Node Installation

Others 2022-04-21 07:40:45 views: 0

planning:

Three physical servers are formed (quorum). For high availability clusters, you can use any odd number higher than 3. For example, if you set up 5 servers, the cluster can handle two failed nodes, etc.

The physical server needs to have inbound connections on ports 2888 , 3888 and 2181 open. If IPtables or Firewall is enabled, make sure to enable the specified ports as zookeeper needs to communicate through these ports.

OS: Centos 7.4 x64
Zookeeper-3.4.10

In this tutorial, we will deploy the zookeeper distributed cluster on the following 3 servers:

10.10.204.63
10.10.204.64
10.10.204.65

prerequisites:

Before installing Zookeeper, you should have JDK (Oracle Java8) installed and configured in your system, which will work with Zookeeper.

" Linux JAVA JDK JRE environment variable installation and configuration "

Step 1: Install Zookeeper on each instance.

Download Zookeeper
 # cd /tmp
 # wget http://apache.fayea.com/zookeeper/stable/zookeeper-3.4.10.tar.gz
 unzip it
 # tar zxvf zookeeper-3.4.10.tar.gz
 Move Zookeeper to /usr/local/
 # mv zookeeper-3.4.10 /usr/local/
 Create soft link
 # ln -s /usr/local/zookeeper-3.4.10 /usr/local/zookeeper
 copy configuration file
 # cp /usr/local/zookeeper/conf/zoo_sample.cfg /usr/local/zookeeper/conf/zoo.cfg
 Create data and log storage directory
 # mkdir -p /usr/local/zookeeper/data
 # mkdir -p /usr/local/zookeeper/logs
 new user
 # groupadd zookeeper
 # useradd -g zookeeper -s /sbin/nologin zookeeper
 Give Zookeeper directory permissions
 # chown -R zookeeper:zookeeper /usr/local/zookeeper-3.4.10 /usr/local/zookeeper
 # chmod +755 /usr/local/zookeeper-3.4.10

Step 2: Modify the configuration file.

# vim /usr/local/zookeeper/conf/zoo.cfg

Defaults:

# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
dataDir=/tmp/zookeeper
# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to “0” to disable auto purge feature
#autopurge.purgeInterval=1

change into:

#The time interval between servers or between the client and the server to maintain the heartbeat, and a heartbeat will be sent every tickTime.
tickTime=2000 #This
configuration item is used to configure Zookeeper to accept clients (the client mentioned here is not the client that the user connects to the Zookeeper server, but the Follower server connected to the Leader in the Zookeeper server cluster) The longest time it takes to initialize the connection How many heartbeat intervals to tolerate. When the Zookeeper server has not received the return information from the client after more than 10 heartbeats (that is, tickTime), it indicates that the client connection failed. The total time length is 10*2000=20 seconds.
initLimit=10 #This
configuration item identifies the length of the message sent between the Leader and the Follower, the length of the request and response time, the longest time length cannot exceed the length of how many tickTimes, and the total time length is 5*2000=10 seconds (applicable to versions above 3.4 ).
syncLimit=5 #This
parameter is used in conjunction with the above parameters. This parameter specifies the number of files to keep. The default is to keep 3.
autopurge.snapRetainCount=3 #This
parameter specifies the cleaning frequency, the unit is hour, you need to fill in an integer of 1 or greater, the default is 0, which means that the self-cleaning function is not enabled (applicable to versions 3.4 and above).
autopurge.purgeInterval=1
maxClientCnxns=60 #Modify
the data directory (can be any directory).
dataDir=/usr/local/zookeeper/data #Add
log directory (can be any directory).
dataLogDir=/usr/local/zookeeper/logs
#The port on which the Zookeeper server listens to accept client access requests.
clientPort=2181 #Add
the following content.
server.1=10.10.204.63:2888:3888
server.2=10.10.204.64:2888:3888
server.3=10.10.204.65:2888:3888

Step 3: Create myid files in each Zookeeper instance respectively.

# echo "1" >> /usr/local/zookeeper/data/myid
 # echo "2" >> /usr/local/zookeeper/data/myid
 # echo "3" >> /usr/local/zookeeper/data/myid

Step 4: Add System Variables.

Edit: /etc/profile file, add the following:

export ZOOKEEPER_HOME=/usr/local/zookeeper/
 export PATH=$ZOOKEEPER_HOME/bin:$PATH

Execute the following command to make the system variable permanent:

# source /etc/profile

Step 5: Create the system unit file.

Create zookeeper.service in the /usr/lib/systemd/system directory and fill in the following:

[Unit]
 Description=zookeeper.service
 After=network.target

[Service]
 Type=forking
 Environment=ZOO_LOG_DIR=/usr/local/zookeeper/
 ExecStart=/usr/local/zookeeper/bin/zkServer.sh start
 ExecStop=/usr/local/zookeeper/bin/zkServer.sh stop
 ExecReload=/usr/local/zookeeper/bin/zkServer.sh restart
 Restart=always
 User=zookeeper
 Group=zookeeper

[Install]
 WantedBy=multi-user.target

Step 6: Start Zookeeper.

Reload the configuration information: systemctl daemon-reload
Start the zookeeper service: systemctl start zookeeper.service
Close the zookeeper service: systemctl stop zookeeper.service
View the process status and log: systemctl status zookeeper.service
Self-start at boot: systemctl enable zookeeper.service
Close self-start :systemctl disable zookeeper.service

Step 7: Release ports 2888, 3888, and 2181.

# firewall-cmd --permanent --zone=public --add-port=2888/tcp
 # firewall-cmd --permanent --zone=public --add-port=3888/tcp
 # firewall-cmd --permanent --zone=public --add-port=2181/tcp

Reload firewall:

# firewall-cmd --reload

Step 8: View Zookeeper Status

Check whether the running status of the three servers is normal.

Check out the 10.10.204.63 node;

[root@10-10-204-63 ~]# /usr/local/zookeeper/bin/zkServer.sh status
 ZooKeeper JMX enabled by default
 Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Mode: leader

Check out the 10.10.204.64 node;

[root@10-10-204-64 ~]# /usr/local/zookeeper/bin/zkServer.sh status
 ZooKeeper JMX enabled by default
 Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Mode: follower

Check out the 10.10.204.65 node;

[root@10-10-204-65 ~]# /usr/local/zookeeper/bin/zkServer.sh status
 ZooKeeper JMX enabled by default
 Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
 Mode: follower

ZooKeeper in the server plays different roles, one will be in the leader (leader) position, and the other two will be in the follower (follower). If you get the same result, then you have the ZooKeeper cluster server installed and configured correctly.

Step 9: Use a client connection on any of the 3 physical servers.

Client connection information is as follows:

[root@10-10-204-63 ~]# /usr/local/zookeeper/bin/zkCli.sh -server 10.10.204.64:2181
 Connecting to 10.10.204.64:2181
 2017-08-13 20:30:11,816 [myid:] - INFO [main:Environment@100] - Client environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT
 2017-08-13 20:30:11,863 [myid:] - INFO [main:Environment@100] - Client environment:host.name=103-28-204-63.10.10.204.63
 2017-08-13 20:30:11,863 [myid:] - INFO [main:Environment@100] - Client environment:java.version=1.8.0_144
 2017-08-13 20:30:11,875 [myid:] - INFO [main:Environment@100] - Client environment:java.vendor=Oracle Corporation
 2017-08-13 20:30:11,883 [myid:] - INFO [main:Environment@100] - Client environment:java.home=/usr/java/jdk1.8.0_144/jre
 2017-08-13 20:30:11,883 [myid:] - INFO [main:Environment@100] - Client environment:java.class.path=/usr/local/zookeeper/bin/../build/classes:/usr/local/zookeeper/bin/../build/lib/*.jar:/usr/local/zookeeper/bin/../lib/slf4j-log4j12-1.6.1.jar:/usr/local/zookeeper/bin/../lib/slf4j-api-1.6.1.jar:/usr/local/zookeeper/bin/../lib/netty-3.10.5.Final.jar:/usr/local/zookeeper/bin/../lib/log4j-1.2.16.jar:/usr/local/zookeeper/bin/../lib/jline-0.9.94.jar:/usr/local/zookeeper/bin/../zookeeper-3.4.10.jar:/usr/local/zookeeper/bin/../src/java/lib/*.jar:/usr/local/zookeeper/bin/../conf:.:/usr/java/jdk1.8.0_144/jre/lib/rt.jar:/usr/java/jdk1.8.0_144/lib/dt.jar:/usr/java/jdk1.8.0_144/lib/tools.jar:/usr/java/jdk1.8.0_144/jre/lib
 2017-08-13 20:30:11,884 [myid:] - INFO [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
 2017-08-13 20:30:11,884 [myid:] - INFO [main:Environment@100] - Client environment:java.io.tmpdir=/tmp
 2017-08-13 20:30:11,884 [myid:] - INFO [main:Environment@100] - Client environment:java.compiler=
 2017-08-13 20:30:11,884 [myid:] - INFO [main:Environment@100] - Client environment:os.name=Linux
 2017-08-13 20:30:11,884 [myid:] - INFO [main:Environment@100] - Client environment:os.arch=amd64
 2017-08-13 20:30:11,885 [myid:] - INFO [main:Environment@100] - Client environment:os.version=3.10.0-514.21.2.el7.x86_64
 2017-08-13 20:30:11,885 [myid:] - INFO [main:Environment@100] - Client environment:user.name=root
 2017-08-13 20:30:11,885 [myid:] - INFO [main:Environment@100] - Client environment:user.home=/root
 2017-08-13 20:30:11,885 [myid:] - INFO [main:Environment@100] - Client environment:user.dir=/root
 2017-08-13 20:30:11,893 [myid:] - INFO [main:ZooKeeper@438] - Initiating client connection, connectString=10.10.204.64:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@69d0a921
 Welcome to ZooKeeper!
 2017-08-13 20:30:12,103 [myid:] - INFO [main-SendThread(10.10.204.64:2181):ClientCnxn$SendThread@1032] - Opening socket connection to server 10.10.204.64/10.10.204.64:2181. Will not attempt to authenticate using SASL (unknown error)
 JLine support is enabled
 2017-08-13 20:30:12,768 [myid:] - INFO [main-SendThread(10.10.204.64:2181):ClientCnxn$SendThread@876] - Socket connection established to 10.10.204.64/10.10.204.64:2181, initiating session
 2017-08-13 20:30:12,935 [myid:] - INFO [main-SendThread(10.10.204.64:2181):ClientCnxn$SendThread@1299] - Session establishment complete on server 10.10.204.64/10.10.204.64:2181, sessionid = 0x15dda7deb6c0000, negotiated timeout = 30000

WATCHER::

WatchedEvent state:SyncConnected type:None path:null
 [zk: 10.10.204.64:2181(CONNECTED) 2] create /renwoledb 'renwole' #Create a data node
 Created /renwoledb
 [zk: 10.10.204.64:2181(CONNECTED) 3] get /renwoledb #Remove node data
 renwole
 cZxid = 0x500000002
 ctime = Sun Aug 13 21:19:24 CST 2017
 mZxid = 0x500000002
 mtime = Sun Aug 13 21:19:24 CST 2017
 pZxid = 0x500000002
 cversion = 0
 dataVersion = 0
 aclVersion = 0
 ephemeralOwner = 0x0
 dataLength = 7
 numChildren = 0

The entire zookeeper cluster has been built and tested so far. If the leader node fails, other followers (followers) will vote for a new leader, so this is what we want for a distributed cluster of Zookeeper.

Please indicate the address of this article when reprinting.

Copyright statement: The original articles on this site are welcome to reprint in any form. 
Please indicate: Install and configure Apache Zookeeper distributed cluster on Linux multi-node |

Guess you like

Origin http://43.154.161.224:23101/article/api/json?id=324525857&siteId=291194637
Recommended
Ranking
Solve the problem that the Chinese display of the drawing using python matplotlib under the Linux system is displayed as a box
[Conference Call for Papers] The 5th International Conference on Civil Engineering, Environmental Resources and Energy Materials (CCESEM 2023)
4-yl Fast Fourier Transform
DataGirdView interlaced display color
[Docker implements test deployment CI/CD----Dingding alarm after the build is successful (7)]
[Asp.net core series] 6 the complete structure of a project in actual combat
The new version of OpenCenterV3 management background
Why can box plots detect outliers and what is the principle?
[Switch] jumbo frame Introduction
C++对象移动（3）——移动构造函数
Daily
More
2024-05-05(0)
2024-05-04(18)
2024-05-03(8)
2024-05-02(0)
2024-05-01(4)
2024-04-30(36)
2024-04-29(5)
2024-04-28(12)
2024-04-27(29)
2024-04-26(22)

Code World

© 2018 codetd.com