RocketMQ asynchronous replication cluster deployment and installation

Anti-counterfeiting code: If you are going to Lingyun Zhi in the future, dare to laugh Huang Chao and her husband.

1. RacketMQ multiple deployment modes

RocketMQ Broker cluster deployment methods are mainly the following: (Slave is not writable, but readable)
1.1. A single Master
is more risky. Once the Broker restarts or goes down, the entire service will be unavailable. Online is not recommended Environmental use.
1.2. In multi-master mode,
one cluster has no slave, and all are masters, such as 2 masters or 3 masters.

Advantages: simple configuration, single Master downtime or restart maintenance has no impact on the application. When the disk configuration is RAID10, even if the machine is down and unrecoverable, the RAID10 disk is very reliable, and the message will not be lost (asynchronous flashing is lost) For a small amount of messages, one will not be lost if you sync the disk). The highest performance.

Disadvantages: During the downtime of a single machine, unconsumed messages on this machine cannot be subscribed until the machine is restored, and the real-time nature of the messages will be affected.

Advantages: Even if the disk is damaged, the message loss is very small, and the real-time message will not be affected, because after the Master is down, consumers can still consume from the Slave, this process is transparent to the application. No human intervention is required. Performance is almost the same as multi-master mode.

Disadvantages: Master is down, the disk is damaged, and a small amount of messages will be lost.

# Start NameServer first, for example, the machine IP is: "10.162.23.70:9876; 10.162.23.71:9876"
# nohup ./bin/mqnamesrv> / dev / null 2> & 1 &
# On machine A, start the first Master
# nohup ./bin/mqbroker -n “10.162.23.70:9876;10.162.23.71:9876” -c ./conf/2m-2s-async/broker-a.properties> / dev / null 2> & 1 &
# on the machine B, start the second Master
# nohup ./bin/mqbroker -n “10.162.23.70:9876; 10.162.23.71:9876” -c ./conf/2m-2s-async/broker-b.properties> / dev / null 2> & 1 & #On
machine C, start the first Slave
nohup ./bin/mqbroker -n “10.162.23.70:9876;10.162.23.71:9876” -c ./conf/2m-2s-async/broker- as.properties> / dev / null 2> & 1 &
# On machine D, start the second Slave
# nohup ./bin/mqbroker -n “10.162.23.70:9876; 10.162.23.71:9876” -c ./conf/ 2m-2s-async / broker-bs.properties> / dev / null 2> & 1 &
1.4. Multiple Master and Multiple Slave Mode, Synchronous Double Write
Each master is configured with a slave, there are multiple pairs of master-slave, HA uses synchronous double write mode, the master and backup are written successfully, and the success is returned to the application.

Advantages: There is no single point of data and service, no message delay when the Master is down, and service availability and data availability are very high

Disadvantages: The performance is slightly lower than the asynchronous replication mode, about 10% lower, and the RT for sending a single message is slightly higher. At present, after the main machine is down, the standby machine cannot be automatically switched to the host machine, and the automatic switching function will be supported later.

# Start NameServer first, for example, the machine IP is: "10.162.23.70:9876; 10.162.23.71:9876"
# nohup ./bin/mqnamesrv> / dev / null 2> & 1 &
# On machine A, start the first Master
# nohup ./bin/mqbroker -n “10.162.23.70:9876;10.162.23.71:9876” -c ./conf/2m-2s-sync/broker-a.properties> / dev / null 2> & 1 &
# on the machine B, start the second Master
# nohup ./bin/mqbroker -n “10.162.23.70:9876; 10.162.23.71:9876” -c ./conf/2m-2s-sync/broker-b.properties> / dev / null 2> & 1 &
# On machine C, start the first Slave
# nohup ./bin/mqbroker -n “10.162.23.70:9876;10.162.23.71:9876” -c ./conf/2m-2s-sync/broker -as.properties> / dev / null 2> & 1 &
# On machine D, start the second Slave
# nohup ./bin/mqbroker -n "10.162.23.70:9876; 10.162.23.71:9876" -c ./conf /2m-2s-sync/broker-bs.properties> / dev / null 2> & 1 &

Note: The
above Broker and Slave pairing is done by specifying the same brokerName parameter. The BrokerId of the Master must be 0, and the BrokerId of the Slave must be a number greater than 0. Multiple Slaves can be mounted under another Master. Multiple Slaves under the same Master can be distinguished by specifying different BrokerId.
In addition to this, nameserver also requires clustering. Our environment adopts asynchronous replication mode of multiple nameservers, multiple masters, and multiple slaves

2. Environmental preparation

PIA]~$E}~1J~MIBSVMTT~8R.png

Roles	CPU name	IP	port	THE	Configuration
Nameserver1	INVOICE-HX-01	10.162.23.70	9876	Redhat7.2	16C32G100G
Nameserver2	INVOICE-HX-02	10.162.23.71	9876	Redhat7.2	16C32G100G
Master1	INVOICE-HX-01	10.162.23.70	10911 10922	Redhat7.2	16C32G100G
Master2	INVOICE-HX-02	10.162.23.71	10911 10922	Redhat7.2	16C32G100G
Slave1	INVOICE-HX-03	10.162.23.72	10911 10922	Redhat7.2	16C32G100G
Slave2	INVOICE-HX-04	10.162.23.73	10911 10922	Redhat7.2	16C32G100G

Moosefs preparation before installation:
# RocketMQ software package: alibaba-rocketmq-3.2.6.tar.gz
# JDK 1.7.79
# SELINUX ： disable
# Installation path: / opt / server / rocketmq
# Log path: / opt / var / logs / rocketmq

# All host configuration hosts
# cat / etc / hosts
10.162.23.70 INVOICE-HX-01
10.162.23.71 INVOICE-HX-02
10.162.23.72 INVOICE-HX-03
10.162.23.72 INVOICE-HX-04

Note: The host resolution must be the real hostname

3. RocketMQ multiple M multiple asynchronous replication cluster installation

3.1.JDK installation
# Installation on all roles
# Upload jdk-7u79-linux-x64.gz
# mkdir / opt / server -p
# tar zvxf jdk-7u79-linux-x64.gz -C / opt / server

# 修改 /etc/profile
# vi /etc/profile
# jdk
JAVA_HOME=/opt/server/jdk1.7.0_79
CLASSPATH=$JAVA_HOME/jre/lib/rt.jar:.
PATH=$JAVA_HOME/bin:$PATH
export JAVA_HOME PATH CLASSPATH

# source /etc/profile
# 检查java版本：
# java -version
java version "1.7.0_79"
Java(TM) SE Runtime Environment (build 1.7.0_79-b06)
Java HotSpot(TM) 64-Bit Server VM (build 20.45-b01, mixed mode)

3.2.RocketMQ extracting installation
# installation on all roles
# upload-rocketmq-3.2.6.tar.gz alibaba
# tar zxvf alibaba-rocketmq-3.2.6.tar.gz
# alibaba-rocketmq mv / opt / Server / rocketmq
# installation The file under the package bin has no execute permission
# chmod + x / opt / server / rocketmq / bin / *

# Create data storage directory
# mkdir -p / opt / data / rocketmq / data / store / commitlog
# Create log directory
# mkdir -p / opt / var / logs / rocketmq /
# Modify log directory
# cd / opt / server / rocketmq / conf
# sed -i 's # $ {user.home} / logs # / opt / var / logs / rocketmq # g' * .xml
# View modification
# grep logs * .xml
3.3.RocketMQ Nameserver startup
# nohup / opt / server / rocketmq / bin / mqnamesrv &
# verify
# cat nohup.out
The Name Server boot success.
# netstat -anlp | grep 9876
tcp 0 0 ::: 9876 ::: * LISTEN 23750 / java

# Boot start
# cat /etc/rc.local
# RocketMQ nameserver
nohup / opt / server / rocketmq / bin / mqnamesrv &
3.4.RocketMQ Master1 Slave1
# Modify master1 (10.162.23.70) configuration
# vim / opt / server / rocketmq / conf /2m-2s-async/broker-a.properties
brokerClusterName = DefaultCluster
brokerName = broker-a
brokerId = 0
namesrvAddr = 10.162.23.70: 9876; 10.162.23.71:9876
defaultTopicQueueNums = 4
autoCreateTopicEnable = true
autoCreateSubscriptionGroup = true
listenPort = 10911
deleteWhen = 04
fileReservedTime = 120
mapedFileSizeCommitLog = 1073741824
mapedFileSizeConsumeQueue = 50000000
destroyMapedFileIntervalForcibly = 120000
redeleteHangedFileInterval = 120000
diskMaxUsedSpaceRatio = 88

storePathRootDir=/opt/data/rocketmq/data/store
storePathCommitLog=/opt/data/rocketmq/data/store/commitlog
maxMessageSize=65536
flushCommitLogLeastPages=4
flushConsumeQueueLeastPages=2
flushCommitLogThoroughInterval=10000
flushConsumeQueueThoroughInterval=60000

checkTransactionMessageEnable = false
sendMessageThreadPoolNums = 128
pullMessageThreadPoolNums = 128

brokerRole=ASYNC_MASTER
flushDiskType=ASYNC_FLUSH

# 启动master1
# nohup /opt/server/rocketmq/bin/mqbroker -c /opt/server/rocketmq/conf/2m-2s-async/broker-a.properties &
# netstat -anlp|grep java
tcp        0      0 :::10911                    :::*                        LISTEN      24220/java
tcp        0      0 :::10912                    :::*                        LISTEN      24220/java

# Startup (nameserver must be started, it is recommended to start manually)
# cat /etc/rc.local
# RocketMQ master-a
nohup / opt / server / rocketmq / bin / mqbroker -c / opt / server / rocketmq / conf / 2m -2s-async / broker-a.properties &

# Modify slave1 (10.162.23.72) configuration
# vim /opt/server/rocketmq/conf/2m-2s-async/broker-a.properties
brokerClusterName = DefaultCluster
brokerName = broker-a
brokerId = 1
namesrvAddr = 10.162.23.70: 9876; 10.162.23.71:9876
defaultTopicQueueNums = 4
autoCreateTopicEnable = true
autoCreateSubscriptionGroup = true
listenPort = 10911
deleteWhen = 04
fileReservedTime = 120
mapedFileSizeCommitLog = 1073741824
mapedFileSizeConsumeQueue = 50000000
destroyMapedFileIntervalForcibly = 120000
redeleteHangedFileInterval = 120000
diskMaxd

checkTransactionMessageEnable = false
sendMessageThreadPoolNums = 128
pullMessageThreadPoolNums = 128

brokerRole=SLAVE
flushDiskType=ASYNC_FLUSH

# Start salve1
# nohup / opt / server / rocketmq / bin / mqbroker -c /opt/server/rocketmq/conf/2m-2s-async/broker-a.properties &

# netstat -anlp|grep java
tcp 0 0 :::10911 :::* LISTEN 23774/java
tcp 0 0 :::10912 :::* LISTEN 23774/java

# Start up (nameserver must be started, it is recommended to start manually)
# cat /etc/rc.local
# RocketMQ slave-a
nohup / opt / server / rocketmq / bin / mqbroker -c / opt / server / rocketmq / conf / 2m -2s-async / broker-a.properties &

3.5.RocketMQ Master2 Slave2
# Modify master2 (10.162.23.71) configuration
# vim /opt/server/rocketmq/conf/2m-2s-async/broker-b.properties
brokerClusterName = DefaultCluster
brokerName = broker-b
brokerId = 0
namesrvAddr = 10.162 .23.70: 9876; 10.162.23.71: 9876
defaultTopicQueueNums = 4
autoCreateTopicEnable = true
autoCreateSubscriptionGroup = true
listenPort = 10911
deleteWhen = 04
fileReservedTime = 120
mapedFileSizeCommitLog = 1073741824
mapedFileSizeConsumeQueue = 50000000
destroyMapedFileIntervalForcibly = 120000
redeleteHangedFile =
MaxMax

checkTransactionMessageEnable = false
sendMessageThreadPoolNums = 128
pullMessageThreadPoolNums = 128

brokerRole=ASYNC_MASTER
flushDiskType=ASYNC_FLUSH

# Start master2
# nohup / opt / server / rocketmq / bin / mqbroker -c /opt/server/rocketmq/conf/2m-2s-async/broker-b.properties &

# 验证
# cat nohup.out
The Name Server boot success.
load config properties file OK, /opt/server/rocketmq/conf/2m-2s-sync/broker-b.properties
The broker[broker-a, 10.162.23.71:10911] boot success.
# netstat -anlp|grep java
tcp        0      0 :::10911                    :::*                        LISTEN      24220/java
tcp        0      0 :::10912                    :::*                        LISTEN      24220/java

# Startup (nameserver must be started, it is recommended to start manually)
# cat /etc/rc.local
# RocketMQ master-b
nohup / opt / server / rocketmq / bin / mqbroker -c / opt / server / rocketmq / conf / 2m -2s-async / broker-b.properties &

# Modify slave2 (10.162.23.73) configuration
# vim /opt/server/rocketmq/conf/2m-2s-async/broker-b.properties
brokerClusterName = DefaultCluster
brokerName = broker-b
brokerId = 1
namesrvAddr = 10.162.23.70: 9876; 10.162.23.71:9876
defaultTopicQueueNums = 4
autoCreateTopicEnable = true
autoCreateSubscriptionGroup = true
listenPort = 10911
deleteWhen = 04
fileReservedTime = 120
mapedFileSizeCommitLog = 1073741824
mapedFileSizeConsumeQueue = 50000000
destroyMapedFileIntervalForcibly = 120000
redeleteHangedFileInterval = 120000
diskMaxd

checkTransactionMessageEnable = false
sendMessageThreadPoolNums = 128
pullMessageThreadPoolNums = 128

brokerRole=SLAVE
flushDiskType=ASYNC_FLUSH

# Start salve1
# nohup / opt / server / rocketmq / bin / mqbroker -c /opt/server/rocketmq/conf/2m-2s-async/broker-b.properties &

# 验证
# cat nohup.out
load config properties file OK, /opt/server/rocketmq/conf/2m-2s-sync/broker-b.properties
The broker[broker-b, 10.162.23.73:10911] boot success. and name server is 10.162.23.70:9876;10.162.23.71:9876
# netstat -anlp|grep java
tcp 0 0 :::10911 :::* LISTEN 23774/java
tcp 0 0 :::10912 :::* LISTEN 23774/java

# Startup (nameserver must be started, it is recommended to start manually)
# cat /etc/rc.local
# RocketMQ slave-b
nohup / opt / server / rocketmq / bin / mqbroker -c / opt / server / rocketmq / conf / 2m -2s-async / broker-b.properties &

Note: The method of kill java process is directly used to stop the service

3.6. RocketMQ nameserver interface
10.162.23.70:9876; 10.162.23.71:9876

RocketMQ asynchronous replication cluster deployment and installation

Guess you like