【RocketMQ】详解如何搭建一个多主多从的RocketMQ集群

Others 2021-12-10 23:36:50 views: null

文章目录

下载二进制包

下载链接：RocketMQ官网

注：二进制包是编译好的，拿来可以直接用，如果不需要定制化，改动源码，可选择二进制包

机器准备

准备三台机器，每台机器需要安装好jdk，1.8以上版本；

集群使用交叉部署：三台机器，每台各自装一个nameserver，三个broker：一个master，两个其他master的slave；

目录结构：

机器1：namesever-1、master-a、b-slave1、c-slave1

机器2：namesever-2、master-b、a-slave1、c-slave2

机器3：namesever-3、master-c、a-slave2、b-slave2

注意：将二进制包解压后的文件，复制到以上12个文件夹中，使用独立的二进制包进行部署，因为需要将各自的日志文件、数据文件单独存储，如果不需要分离，则可使用同一个包

部署nameserver

创建两个目录：data、logs，用于存放数据和日志，rocketmq471是提前解压好的二进制文件目录：

修改配置文件

RocketMQ主目录：/neworiental/rocketmq01/rocketmq-nameserver/rocketmq471

新建nameserver配置文件：/neworiental/rocketmq01/rocketmq-nameserver/rocketmq471/conf/nameserver.conf

修改nameserver.conf文件：

#数据保存目录,需要配置,不配置的话默认在/root/namesrv
kvConfigPath=/neworiental/rocketmq01/rocketmq-nameserver/data/kvConfig.json
#是否启用Epoll IO模型
useEpollNativeSelector=true
#端口
listenPort=9876

修改日志路径

日志文件路径：/neworiental/rocketmq01/rocketmq-nameserver/rocketmq471/conf/logback_namesrv.xml

使用命令批量替换：

sed -i ‘s?${user.home}/logs/rocketmqlogs?/neworiental/rocketmq01/rocketmq-nameserver/logs?’ logback_namesrv.xml

打开logback_namesrv.xml，可以看到路径已全部替换：

修改启动脚本

可根据需求修改启动脚本，如：JVM配置、堆栈大小等，路径：

/neworiental/rocketmq01/rocketmq-nameserver/rocketmq471/bin/mqnamesrv

启动nameserver

nohup sh /neworiental/rocketmq01/rocketmq-nameserver/rocketmq471/bin/mqnamesrv -c /neworiental/rocketmq01/rocketmq-nameserver/rocketmq471/conf/nameserver.conf >/dev/null 2>&1 &

根据以上步骤启动剩下两个nameserver，配置文件直接复制使用

部署broker

以master-a节点为示例

修改日志路径

日志配置文件路径：

/neworiental/rocketmq01/rocketmq-a/rocketmq471/conf/logback_broker.xml

/neworiental/rocketmq01/rocketmq-a/rocketmq471/conf/logback_namesrv.xml

/neworiental/rocketmq01/rocketmq-a/rocketmq471/conf/logback_tools.xml

执行命令：（注意不同的Broker目录不一样）

sed -i 's?${user.home}/logs/rocketmqlogs?/neworiental/rocketmq01/rocketmq-a/logs?' logback_broker.xml
sed -i 's?${user.home}/logs/rocketmqlogs?/neworiental/rocketmq01/rocketmq-a/logs?' logback_namesrv.xml
sed -i 's?${user.home}/logs/rocketmqlogs?/neworiental/rocketmq01/rocketmq-a/logs?' logback_tools.xml

修改配置文件

配置文件路径：

RoketMQ的端口配置需要注意，详情见：集群端口配置讲解

#端口（注意：broker启动后，会占用3个端口，分别在listenPort基础上-2，+1，供内部程序使用，所以集群一定要规划好端口，避免冲突）
listenPort = 10911
#slave连接端口,配置之后无效，所以不用配
#haListenPort = 20911
#RocketMQ主目录
rocketmqHome = /neworiental/rocketmq01/rocketmq-a/rocketmq471
#集群名称
brokerClusterName = Rocketmq01
#nameserver地址，多个以分号隔开
namesrvAddr=172.22.36.68:9876;172.22.36.69:9876;172.22.36.70:9876
#broker名称（每组主从节点使用相同的brokerName，方便查看）
brokerName = broker-a
#Broker对外提供服务地址
brokerIP1 = 172.22.36.68
#供slave同步消息的地址
brokerIP2= 172.22.36.68
#broker的id,0为主,正数0为从
brokerId = 0
#数据存放的根目录
storePathRootDir = /neworiental/rocketmq01/rocketmq-a/store
#commit log保存目录
storePathCommitLog = /neworiental/rocketmq01/rocketmq-a/store/commitlog
#消费队列存储路径存储路径
storePathConsumerQueue = /neworiental/rocketmq01/rocketmq-a/store/consumequeue
#删除文件时间点，默认凌晨4点。24小时制，单位小时
deleteWhen = 04
#文件保存小时数
fileReservedTime = 168
#SYNC_MASTER/ASYNC_MASTER/SLAVE 节点角色
brokerRole = ASYNC_MASTER
#刷盘方式
flushDiskType = ASYNC_FLUSH
#是否允许Broker自动创建Topic
autoCreateTopicEnable = false
#是否允许 Broker 自动创建订阅组
autoCreateSubscriptionGroup = false
#linux开启epoll
useEpollNativeSelector = true

修改启动脚本

文件路径：/neworiental/rocketmq01/rocketmq-a/rocketmq471/bin/runbroker.sh

可根据需求，修改jvm参数、GC日志目录等：

启动Broker

nohup sh /neworiental/rocketmq01/rocketmq-a/rocketmq471/bin/mqbroker -c /neworiental/rocketmq01/rocketmq-a/rocketmq471/conf/broker.conf >/dev/null 2>&1 &

根据以上步骤，部署其余8个broker。

友情提示：对于这种中间件集群的搭建，基本就是不同的节点，要设置不同的存储目录，不同的配置文件，由于节点比较多，操作比较繁琐，很容易出现失误，所以可以先搭建好一个节点，确保可行之后，在本地模拟线上每个机器、每个节点的目录结构，把所有的节点对应的配置文件先改好了、检查好了，最后再每个节点逐个上传、启动，一定会事半功倍！

部署完成之后，可以进入任意broker的RocketMQ二进制包的bin目录，用命令查看集群信息：

sh mqadmin clusterList -n 127.0.0.1:9876

可以看到3主3从的RocketMQ集群搭建成功！

部署Console

RocketMQ-Console是RocketMQ的控制台页面，方便查看集群的具体信息和使用情况

下载源码

地址：https://github.com/apache/rocketmq-externals.git

下载后，使用IDEA打开rocketmq-console子模块

修改配置文件

修改日志路径

检查pom.xml，修改RocketMQ版本：

设置用户

需在application.properties中开启用户登录 rocketmq.config.loginRequired=true：（如果不需要设置用户登录，可忽略）

users.properties文件需要拷贝一份到console的data目录，方便修改，热加载：/neworiental/rocketmq01/rocketmq-console/data

打包部署

使用maven命令或IDEA将console打包，生成jar文件：rocketmq-console-ng-2.0.0.jar

上传到服务器后，java -jar rocketmq-console-ng-2.0.0.jar 启动服务即可

部署Exporter

需下载源码，自己打包：https://github.com/apache/rocketmq-exporter

exporter部署一个就可以

下载后，修改几个地方

nameserver地址：

日志目录：

pom.xml 修改rocketmq版本

这段注释掉：

脚本：

shutdown.sh

#!/bin/bash
. /etc/profile

PID=`ps -ef | grep '/neworiental/rocketmq02/rocketmq-exporter' | grep -v grep | awk '{print $2}'`
if [[ "" !=  "$PID" ]]; then
  echo "killing rocketmq-console : $PID"
  kill $PID
fi

start.sh

#!/bin/sh
. /etc/profile

LOGS_DIR=/neworiental/rocketmq02/rocketmq-expoter/logs

MAIN_JAR="-jar /neworiental/rocketmq02/rocketmq-expoter/rocketmq-exporter-0.0.2-SNAPSHOT.jar  "
JAVA_ARGS="-server -Xms4096m -Xmx4096m -XX:NewSize=1500m -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=70 -XX:+PrintGCDetails -XX:+PrintHeapAtGC  -XX:ThreadStackSize=512 -Xloggc:${LOGS_DIR}/gc.log "

if [ ! -d ${LOGS_DIR} ]
then
    mkdir -p ${LOGS_DIR}
fi
echo ${JAVA_ARGS} ${MAIN_JAR} ${CLASSPATH}
nohup java ${JAVA_ARGS} ${MAIN_JAR} 1>>${LOGS_DIR}/stdout.log 2>>${LOGS_DIR}/stderr.log &
echo "deploying rocketmq-exporter-0.0.2-SNAPSHOT.jar now ..."

restart.sh

#!/bin/bash

PID=`ps -ef | grep '/neworiental/rocketmq02/rocketmq-expoter' | grep -v grep | awk '{print $2}'`
if [[ "" !=  "$PID" ]]; then
  echo "killing rocketmq-exporter : $PID"
  kill $PID
fi

sleep 1

LOGS_DIR=/neworiental/rocketmq02/rocketmq-exporter/logs

MAIN_JAR="-jar /neworiental/rocketmq02/rocketmq-expoter/rocketmq-exporter-0.0.2-SNAPSHOT.jar  "
JAVA_ARGS="-server -Xms4096m -Xmx4096m -XX:NewSize=1500m -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=70 -XX:+PrintGCDetails -XX:+PrintHeapAtGC  -XX:ThreadStackSize=512 -Xloggc:${LOGS_DIR}/gc.log "

if [ ! -d ${LOGS_DIR} ]
then
    mkdir -p ${LOGS_DIR}
fi
echo ${JAVA_ARGS} ${MAIN_JAR} ${CLASSPATH}
nohup java ${JAVA_ARGS} ${MAIN_JAR} 1>>${LOGS_DIR}/stdout.log 2>>${LOGS_DIR}/stderr.log &
echo "deploying rocketmq-exporter-0.0.2-SNAPSHOT.jar now ..."

附录

所有Broker配置文件

机器1：

master-a

listenPort = 10911
rocketmqHome = /neworiental/rocketmq01/rocketmq-a/rocketmq471
brokerClusterName = Rocketmq01
namesrvAddr=172.22.36.68:9876;172.22.36.69:9876;172.22.36.70:9876
brokerName = broker-a
brokerIP1 = 172.22.36.68
brokerIP2= 172.22.36.68
brokerId = 0
storePathRootDir = /neworiental/rocketmq01/rocketmq-a/store
storePathCommitLog = /neworiental/rocketmq01/rocketmq-a/store/commitlog
storePathConsumerQueue = /neworiental/rocketmq01/rocketmq-a/store/consumequeue
deleteWhen = 04
fileReservedTime = 168
brokerRole = ASYNC_MASTER
flushDiskType = ASYNC_FLUSH
autoCreateTopicEnable = false
autoCreateSubscriptionGroup = false
useEpollNativeSelector = true

b-slave1：

listenPort = 10915
rocketmqHome = /neworiental/rocketmq01/rocketmq-b-s1/rocketmq471
brokerClusterName = Rocketmq01
namesrvAddr=172.22.36.68:9876;172.22.36.69:9876;172.22.36.70:9876
brokerName = broker-b
brokerIP1 = 172.22.36.68
brokerIP2 = 172.22.36.68
brokerId = 1
storePathRootDir = /neworiental/rocketmq01/rocketmq-b-s1/store
storePathCommitLog = /neworiental/rocketmq01/rocketmq-b-s1/store/commitlog
storePathConsumerQueue = /neworiental/rocketmq01/rocketmq-b-s1/store/consumequeue
deleteWhen = 04
fileReservedTime = 168
brokerRole = SLAVE
flushDiskType = ASYNC_FLUSH
autoCreateTopicEnable = false
autoCreateSubscriptionGroup = false
useEpollNativeSelector = true

c-slave1：


listenPort = 10919
rocketmqHome = /neworiental/rocketmq01/rocketmq-c-s1/rocketmq471
brokerClusterName = Rocketmq01
namesrvAddr=172.22.36.68:9876;172.22.36.69:9876;172.22.36.70:9876
brokerName = broker-c
brokerIP1 = 172.22.36.68
brokerIP2 = 172.22.36.68
brokerId = 1
storePathRootDir = /neworiental/rocketmq01/rocketmq-c-s1/store
storePathCommitLog = /neworiental/rocketmq01/rocketmq-c-s1/store/commitlog
storePathConsumerQueue = /neworiental/rocketmq01/rocketmq-c-s1/store/consumequeue
deleteWhen = 04
fileReservedTime = 168
brokerRole = SLAVE
flushDiskType = ASYNC_FLUSH
autoCreateTopicEnable = false
autoCreateSubscriptionGroup = false
useEpollNativeSelector = true

机器2：

master-b：


listenPort = 10911
rocketmqHome = /neworiental/rocketmq01/rocketmq-b/rocketmq471
brokerClusterName = Rocketmq01
namesrvAddr=172.22.36.68:9876;172.22.36.69:9876;172.22.36.70:9876
brokerName = broker-b
brokerIP1 = 172.22.36.69
brokerIP2= 172.22.36.69
brokerId = 0
storePathRootDir = /neworiental/rocketmq01/rocketmq-b/store
storePathCommitLog = /neworiental/rocketmq01/rocketmq-b/store/commitlog
storePathConsumerQueue = /neworiental/rocketmq01/rocketmq-b/store/consumequeue
deleteWhen = 04
fileReservedTime = 168
brokerRole = ASYNC_MASTER
flushDiskType = ASYNC_FLUSH
autoCreateTopicEnable = false
autoCreateSubscriptionGroup = false
useEpollNativeSelector = true

a-slave1：


listenPort = 10915
rocketmqHome = /neworiental/rocketmq01/rocketmq-a-s1/rocketmq471
brokerClusterName = Rocketmq01
namesrvAddr=172.22.36.68:9876;172.22.36.69:9876;172.22.36.70:9876
brokerName = broker-a
brokerIP1 = 172.22.36.69
brokerIP2 = 172.22.36.69
brokerId = 1
storePathRootDir = /neworiental/rocketmq01/rocketmq-a-s1/store
storePathCommitLog = /neworiental/rocketmq01/rocketmq-a-s1/store/commitlog
storePathConsumerQueue = /neworiental/rocketmq01/rocketmq-a-s1/store/consumequeue
deleteWhen = 04
fileReservedTime = 168
brokerRole = SLAVE
flushDiskType = ASYNC_FLUSH
autoCreateTopicEnable = false
autoCreateSubscriptionGroup = false
useEpollNativeSelector = true

c-slave2：


listenPort = 10919
rocketmqHome = /neworiental/rocketmq01/rocketmq-c-s2/rocketmq471
brokerClusterName = Rocketmq01
namesrvAddr=172.22.36.68:9876;172.22.36.69:9876;172.22.36.70:9876
brokerName = broker-c
brokerIP1 = 172.22.36.69
brokerIP2= 172.22.36.69
brokerId = 2
storePathRootDir = /neworiental/rocketmq01/rocketmq-c-s2/store
storePathCommitLog = /neworiental/rocketmq01/rocketmq-c-s2/store/commitlog
storePathConsumerQueue = /neworiental/rocketmq01/rocketmq-c-s2/store/consumequeue
deleteWhen = 04
fileReservedTime = 168
brokerRole = SLAVE
flushDiskType = ASYNC_FLUSH
autoCreateTopicEnable = false
autoCreateSubscriptionGroup = false
useEpollNativeSelector = true

机器3：

master-c：


listenPort = 10911
rocketmqHome = /neworiental/rocketmq01/rocketmq-c/rocketmq471
brokerClusterName = Rocketmq01
namesrvAddr=172.22.36.68:9876;172.22.36.69:9876;172.22.36.70:9876
brokerName = broker-c
brokerIP1 = 172.22.36.70
brokerIP2= 172.22.36.70
brokerId = 0
storePathRootDir = /neworiental/rocketmq01/rocketmq-c/store
storePathCommitLog = /neworiental/rocketmq01/rocketmq-c/store/commitlog
storePathConsumerQueue = /neworiental/rocketmq01/rocketmq-c/store/consumequeue
deleteWhen = 04
fileReservedTime = 168
brokerRole = ASYNC_MASTER
flushDiskType = ASYNC_FLUSH
autoCreateTopicEnable = false
autoCreateSubscriptionGroup = false
useEpollNativeSelector = true

a-slave2：


listenPort = 10915
rocketmqHome = /neworiental/rocketmq01/rocketmq-a-s2/rocketmq471
brokerClusterName = Rocketmq01
namesrvAddr=172.22.36.68:9876;172.22.36.69:9876;172.22.36.70:9876
brokerName = broker-a
brokerIP1 = 172.22.36.70
brokerIP1 = 172.22.36.70
brokerId = 2
storePathRootDir = /neworiental/rocketmq01/rocketmq-a-s2/store
storePathCommitLog = /neworiental/rocketmq01/rocketmq-a-s2/store/commitlog
storePathConsumerQueue = /neworiental/rocketmq01/rocketmq-a-s2/store/consumequeue
deleteWhen = 04
fileReservedTime = 168
brokerRole = SLAVE
flushDiskType = ASYNC_FLUSH
autoCreateTopicEnable = false
autoCreateSubscriptionGroup = false
useEpollNativeSelector = true

b-slave2：


listenPort = 10919
rocketmqHome = /neworiental/rocketmq01/rocketmq-b-s2/rocketmq471
brokerClusterName = Rocketmq01
namesrvAddr=172.22.36.68:9876;172.22.36.69:9876;172.22.36.70:9876
brokerName = broker-b
brokerIP1 = 172.22.36.70
brokerIP1 = 172.22.36.70
brokerId = 2
storePathRootDir = /neworiental/rocketmq01/rocketmq-b-s2/store
storePathCommitLog = /neworiental/rocketmq01/rocketmq-b-s2/store/commitlog
storePathConsumerQueue = /neworiental/rocketmq01/rocketmq-b-s2/store/consumequeue
deleteWhen = 04
fileReservedTime = 168
brokerRole = SLAVE
flushDiskType = ASYNC_FLUSH
autoCreateTopicEnable = false
autoCreateSubscriptionGroup = false
useEpollNativeSelector = true

第一次搭建，端口号分配不是很合理，应该每一组主从使用相同的端口号。下面是第二次搭建分配的情况，强迫症看着舒服多了：
Broker

节点 机器 端口
broker-a 172.22.36.68 20911
broker-a-s1 172.22.36.69 20911
broker-a-s2 172.22.36.70 20911
broker-b 172.22.36.69 20915
broker-b-s1 172.22.36.68 20915
broker-b-s2 172.22.36.70 20915
broker-c 172.22.36.70 20919
broker-c-s1 172.22.36.68 20919
broker-c-s2 172.22.36.69 20919

nameserver

节点 机器 端口
nameserver 172.22.36.68 9976
nameserver 172.22.36.69 9976
nameserver 172.22.36.70 9976

console

节点 机器 端口
console 172.22.36.69 8282

后续的话，还可以加入Systemd管理，注意Systemd需要对每个broker单独配置。

Guess you like

Origin blog.csdn.net/sinat_14840559/article/details/108391651
Recommended
LFOSSA Yuanlaisusu Open Course | Mastering the Cloud Native Future: Comprehensive Guide to CNCF Certification and Exam Preparation Tips
Ranking
C++ Basic Syntax
bootstrapTable hides a column based on a condition
Why is reentrant lock recommended instead of Synchronized when dynamic high concurrency?
hexo create a blog
[Fully open source and non-encrypted version] Imitation of the eighth district distribution/online signature/multiple sets of download templates/APP distribution hosting/APP packaging and packaging
Polymerization combination
https://www.flysnow.org/2017/05/06/go-in-action-go-log.html
From the perspective of Flutter and the front-end, talk about how to ensure UI fluency under the single-threaded model
nginx-301, 302 redirect
Geolocation by IP Address in ASP.NET
Daily
More
2024-04-26(22)
2024-04-25(32)
2024-04-24(30)
2024-04-23(30)
2024-04-22(5)
2024-04-21(0)
2024-04-20(6)
2024-04-19(5)
2024-04-18(0)
2024-04-17(31)

Code World

© 2018 codetd.com