分布式消息队列RocketMQ部署与监控（转）

原创作品，允许转载，转载时请务必以超链接形式标明文章原始出处、作者信息和本声明。否则将追究法律责任。 http://sofar.blog.51cto.com/353572/1540874

==========================================================================================

一、RocketMQ简介

==========================================================================================

RocketMQ是一款分布式、队列模型的消息中间件，具有以下特点：

1、支持严格的消息顺序；

2、支持Topic与Queue两种模式；

3、亿级消息堆积能力；

4、比较友好的分布式特性；

5、同时支持Push与Pull方式消费消息；

http://blog.csdn.net/a19881029/article/details/34446629

http://singlefly.blog.51cto.com/4658189/1368579

本文采用双Master的架构模式。

==========================================================================================

二、服务器分布

==========================================================================================

1、相关说明

IP地址	主机名	机型	角色	架构模式
10.217.121.123	rocketmq-master1	128G内存	nameserver、brokerserver	Master1（双Master模式）
10.217.122.31	rocketmq-master2	128G内存	nameserver、brokerserver	Master2（双Master模式）

2、hosts信息添加

# vim /etc/hosts

10.217.121.123 mqnameserver1

10.217.122.31 mqnameserver2

10.217.121.123 rocketmq-master1

10.217.122.31 rocketmq-master2

3、系统环境

CentOS 6.3

4、总体架构

==========================================================================================

三、RocketMQ安装与配置

==========================================================================================

1、JDK安装

http://download.oracle.com/otn-pub/java/jdk/7u65-b17/jdk-7u65-linux-x64.tar.gz

# tar xvzf jdk-7u65-linux-x64.gz -C /usr/local

# rm -f jdk-7u65-linux-x64.gz

# cd /usr/local && ln -s jdk1.7.0_65 jdk

# cd /usr/local/bin && ln -s /usr/local/jdk/bin/java

2、RocketMQ安装

https://github.com/alibaba/RocketMQ/releases

# tar xvzf alibaba-rocketmq-3.1.7.tar.gz -C /usr/local

# mv alibaba-rocketmq alibaba-rocketmq-3.1.7

# ln -s alibaba-rocketmq-3.1.7 rocketmq

# rm alibaba-rocketmq-3.1.7.tar.gz

# ll /usr/local

3、环境变量设置

# vim /etc/profile

export PATH=$PATH:/usr/local/bin

export JAVA_HOME=/usr/local/jdk

export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

export PATH=$PATH:$JAVA_HOME/bin

export ROCKETMQ_HOME=/usr/local/rocketmq

export PATH=$PATH::$ROCKETMQ_HOME/bin

# source /etc/profile

# cd /usr/local/rocketmq/bin && sh os.sh

4、主机名设置

(1)、Master1服务器

# vim /etc/hosts

10.217.121.123 mqnameserver1

10.217.122.31 mqnameserver2

10.217.121.123 rocketmq-master1

10.217.122.31 rocketmq-master2

# sed -i '/HOSTNAME/d' /etc/sysconfig/network

# echo 'HOSTNAME=rocketmq-master1' >> /etc/sysconfig/network

# hostname rocketmq-master1

(2)、Master2服务器

# vim /etc/hosts

10.217.121.123 mqnameserver1

10.217.122.31 mqnameserver2

10.217.121.123 rocketmq-master1

10.217.122.31 rocketmq-master2

# sed -i '/HOSTNAME/d' /etc/sysconfig/network

# echo 'HOSTNAME=rocketmq-master2' >> /etc/sysconfig/network

# hostname rocketmq-master2

5、RocketMQ配置

(1)、Master1服务器

# vim /usr/local/rocketmq/conf/2m-noslave/broker-a.properties

brokerClusterName=AdpMqCluster

brokerName=broker-a

brokerId=0

namesrvAddr=mqnameserver1:9876;mqnameserver2:9876

defaultTopicQueueNums=4

autoCreateTopicEnable=true

autoCreateSubscriptionGroup=true

listenPort=10911

deleteWhen=04

fileReservedTime=120

mapedFileSizeCommitLog=1073741824

mapedFileSizeConsumeQueue=50000000

destroyMapedFileIntervalForcibly=120000

redeleteHangedFileInterval=120000

diskMaxUsedSpaceRatio=88

storePathRootDir=/data/rocketmq/store

storePathCommitLog=/data/rocketmq/store/commitlog

maxMessageSize=65536

flushCommitLogLeastPages=4

flushConsumeQueueLeastPages=2

flushCommitLogThoroughInterval=10000

flushConsumeQueueThoroughInterval=60000

brokerRole=ASYNC_MASTER

flushDiskType=ASYNC_FLUSH

checkTransactionMessageEnable=false

sendMessageThreadPoolNums=128

pullMessageThreadPoolNums=128

(2)、Master2服务器

# vim /usr/local/rocketmq/conf/2m-noslave/broker-b.properties

brokerClusterName=AdpMqCluster

brokerName=broker-b

brokerId=1

namesrvAddr=mqnameserver1:9876;mqnameserver2:9876

defaultTopicQueueNums=4

autoCreateTopicEnable=true

autoCreateSubscriptionGroup=true

listenPort=10911

deleteWhen=04

fileReservedTime=120

mapedFileSizeCommitLog=1073741824

mapedFileSizeConsumeQueue=50000000

destroyMapedFileIntervalForcibly=120000

redeleteHangedFileInterval=120000

diskMaxUsedSpaceRatio=88

storePathRootDir=/data/rocketmq/store

storePathCommitLog=/data/rocketmq/store/commitlog

maxMessageSize=65536

flushCommitLogLeastPages=4

flushConsumeQueueLeastPages=2

flushCommitLogThoroughInterval=10000

flushConsumeQueueThoroughInterval=60000

brokerRole=ASYNC_MASTER

flushDiskType=ASYNC_FLUSH

checkTransactionMessageEnable=false

sendMessageThreadPoolNums=128

pullMessageThreadPoolNums=128

(3)、runbroker.sh参数调整

runbroker.sh需要根据内存大小进行适当地调整

JAVA_OPT_1="-server-Xms80g -Xmx80g -Xmn20g -XX:PermSize=1g -XX:MaxPermSize=1g"

6、服务启动

# mkdir -p /data/rocketmq/store/commitlog /data/logs

# cd /usr/local/rocketmq/conf && sed -i 's#${user.home}#/data#g' *.xml

(1)、启动NameServer【master1、master2】

# cd /usr/local/rocketmq/bin

# nohup sh mqnamesrv &

(2)、启动BrokerServer A【master1】

# cd /usr/local/rocketmq/bin

# nohup sh mqbroker -c ../conf/2m-noslave/broker-a.properties >/dev/null 2>&1 &

(3)、启动BrokerServer B【master2】

# cd /usr/local/rocketmq/bin

# nohup sh mqbroker -c ../conf/2m-noslave/broker-b.properties >/dev/null 2>&1 &

# netstat -ntlp

# jps

==========================================================================================

四、RocketMQ服务健康监控

==========================================================================================

1、依赖组件安装

# yum -y install nmap

2、nameserver监控

# vim /data/scripts/check_nameserver_health.sh

 
         #!/bin/sh 
        
         PATH= 
         /sbin 
         : 
         /bin 
         : 
         /usr/sbin 
         : 
         /usr/bin 
         : 
         /usr/local/bin 
         : 
         /usr/local/sbin 
        
         SRV_PORT= 
         "9876"                     
         ## 端口号 
        
         SRV_PROT= 
         "tcp"                      
         ## 协议类型 
        
         SRV_NAME= 
         "rocketmq_nameserver"      
         ## 服务名 
        
         ## 是否已正确扫描 
        
         SCAN_FLAG=0 
        
         for  
         ETH_NAME  
         in  
         ` 
         /sbin/ifconfig  
         |  
         grep  
         eth1 |  
         awk  
         '{print $1}' 
         ` 
        
         do 
        
         ETH1_ADDR=` 
         /sbin/ifconfig  
         ${ETH_NAME} |  
         awk  
         -F  
         ':'  
         '/inet addr/{print $2}'  
         |  
         sed  
         's/[a-zA-Z ]//g' 
         ` 
        
         [[ -z  
         "${ETH1_ADDR}"  
         ]] &&  
         continue  
         ||  
         break 
        
         done 
        
         TMP_SRV_PROT=` 
         echo  
         ${SRV_PROT} |  
         tr  
         '[A-Z]'  
         '[a-z]' 
         ` 
        
         if  
         [[  
         "${TMP_SRV_PROT}"  
         ==  
         "tcp"  
         ]];  
         then 
        
         PROT_OPT= 
         "S" 
        
         elif  
         [[  
         "${TMP_SRV_PROT}"  
         ==  
         "udp"  
         ]];  
         then 
        
         PROT_OPT= 
         "U" 
        
         else 
        
         echo  
         "未知的协议类型！"  
         && exit1 
        
         fi 
        
         ## 最多扫描3次，成功一次即可，以避免网络抖动而导致误判 
        
         for  
         ((i=0; i<3; i++));  
         do 
        
         RETVAL=` 
         /usr/bin/nmap  
         -n -s${PROT_OPT} -p ${SRV_PORT} ${ETH1_ADDR} |  
         grep  
         open 
         ` 
        
         [[ -n  
         "${RETVAL}"  
         ]] && SCAN_FLAG=1; 
         break  
         ||  
         sleep  
         10 
        
         done 
        
         if  
         [[ ${SCAN_FLAG} - 
         ne  
         1 ]];  
         then 
        
         [[ -n ` 
         ps  
         aux |  
         grep  
         java |  
         grep  
         namesrv` ]] &&  
         kill  
         -9 ` 
         ps  
         aux |  
         grep  
         java |  
         awk  
         '/namesrv/{print $2}' 
         ` 
        
         cd  
         /usr/local/rocketmq/bin  
         &&  
         nohup  
         sh mqnamesrv & 
        
         fi

3、brokerserver监控

# vim /data/scripts/check_brokerserver_health.sh

 
         #!/bin/sh 
        
         PATH= 
         /sbin 
         : 
         /bin 
         : 
         /usr/sbin 
         : 
         /usr/bin 
         : 
         /usr/local/bin 
         : 
         /usr/local/sbin 
        
         SRV_PORT= 
         "10911"                       
         ## 端口号 
        
         SRV_PROT= 
         "tcp"                         
         ## 协议类型 
        
         SRV_NAME= 
         "rocketmq_brokerserver"       
         ## 服务名 
        
         ## 是否已正确扫描 
        
         SCAN_FLAG=0 
        
         for  
         ETH_NAME  
         in  
         ` 
         /sbin/ifconfig  
         |  
         grep  
         eth1 |  
         awk  
         '{print $1}' 
         ` 
        
         do 
        
         ETH1_ADDR=` 
         /sbin/ifconfig  
         ${ETH_NAME} |  
         awk  
         -F  
         ':'  
         '/inet addr/{print $2}'  
         |  
         sed  
         's/[a-zA-Z ]//g' 
         ` 
        
         [[ -z  
         "${ETH1_ADDR}"  
         ]] &&  
         continue  
         ||  
         break 
        
         done 
        
         TMP_SRV_PROT=` 
         echo  
         ${SRV_PROT} |  
         tr  
         '[A-Z]'  
         '[a-z]' 
         ` 
        
         if  
         [[  
         "${TMP_SRV_PROT}"  
         ==  
         "tcp"  
         ]];  
         then 
        
         PROT_OPT= 
         "S" 
        
         elif  
         [[  
         "${TMP_SRV_PROT}"  
         ==  
         "udp"  
         ]];  
         then 
        
         PROT_OPT= 
         "U" 
        
         else 
        
         echo  
         "未知的协议类型！"  
         && exit1 
        
         fi 
        
         ## 最多扫描3次，成功一次即可，以避免网络抖动而导致误判 
        
         for  
         ((i=0; i<3; i++));  
         do 
        
         RETVAL=` 
         /usr/bin/nmap  
         -n -s${PROT_OPT} -p ${SRV_PORT} ${ETH1_ADDR} |  
         grep  
         open 
         ` 
        
         [[ -n  
         "${RETVAL}"  
         ]] && SCAN_FLAG=1; 
         break  
         ||  
         sleep  
         10 
        
         done 
        
         if  
         [[ ${SCAN_FLAG} - 
         ne  
         1 ]];  
         then 
        
         [[ -n ` 
         ps  
         aux |  
         grep  
         java |  
         grep  
         broker` ]] &&  
         kill  
         -9 ` 
         ps  
         aux |  
         grep  
         java |  
         awk  
         '/broker/{print $2}' 
         ` 
        
         cd  
         /usr/local/rocketmq/bin  
         &&  
         nohup  
         sh mqbroker -c .. 
         /conf/2m-noslave/broker-a 
         .properties & 
        
         fi

【注意】

以上红色标注部分，在master2上的名称为“broker-b.properties”

4、crontab信息添加

# touch /var/run/check_rocketmq_nameserver.lock

# touch /var/run/check_rocketmq_brokerserver.lock

# echo 'touch /var/run/check_rocketmq_nameserver.lock' >> /etc/rc.d/rc.local

# echo 'touch /var/run/check_rocketmq_brokerserver.lock' >> /etc/rc.d/rc.local

# crontab -e

*/2 * * * * (flock --timeout=0 /var/run/check_rocketmq_nameserver.lock /data/scripts/check_nameserver_health.sh >/dev/null 2>&1)

*/2 * * * * (flock --timeout=0 /var/run/check_rocketmq_brokerserver.lock /data/scripts/check_brokerserver_health.sh >/dev/null 2>&1)

本文出自 “人生理想在于坚持不懈” 博客，请务必保留此出处http://sofar.blog.51cto.com/353572/1540874

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

附：

https://github.com/alibaba/RocketMQ/releases

分布式消息队列RocketMQ部署与监控 （转）

猜你喜欢

分布式消息队列RocketMQ部署与监控（转）