I. Introduction
RocketMQ is a distributed queue messaging middleware model by Alibaba independent research and development, based on Metaq kernel, was renamed RocketMq after Metaq3.X, draw only reference to the JMS specification, but does not follow the JMS specification. Abandoned by zookeeper, the use of more lightweight nameserver for network routing, improve service performance, and supports the message failed retry mechanism.
In addition to the regular JMS concepts:
nameserver: store all the current cluster information Brokers, Topic correspondence relationship with the Broker.
broker: cluster core module, is responsible Topic message storage, consumer site management (consumption schedule).
Cluster deployment structure:
This article describes only build dual master mode, because the system of concurrent less demanding, dual master mode is sufficient, other cluster model your own Baidu.
Second, workflow
1. Start Nameserver, after Nameserver up listening port, waiting Broker, Produer, Consumer and even up the equivalent of a routing control center (the role of zookeeper, but more lightweight than ZK).
2, Broker start, with all Nameserver holding long connection time to send the heartbeat packet. Broker heartbeat packet contains information about the current (IP + port, etc.), and store all topic information. After successful registration, there Topic mapping relationship with the Broker of Nameserver cluster.
3, before sending and receiving messages, first create a topic, create need to specify the topic on which you want to store Broker when topic. Topic can also be created automatically when sending messages.
4, Producer sends a message, wherein a station to begin to establish a long connection Namesrv cluster startup, and acquires the current transmission is present on the Topic Broker from which Nameserver then establish connections with the corresponding length Broker, message directly to the Broker.
5, Consumer similar with the Producer. With one of them Nameserver establish a long connection, obtain the current subscription Topic presence on which Broker, then establish a connection directly with the channel Broker, began to consume news.
Third, the deployment configuration
rocketmq version is 4.5.2, the downloaded files on a linux server decompression.
In the extraction path below the new store folders (previously part of a new online reference folder, but lead broker failed to start, the cause is still under investigation)
Modified bin / runbroker.sh the JVM configuration, depending on the case may be produced, preferably not less than 1g
JAVA_OPT="${JAVA_OPT} -server -Xms1g -Xmx1g -Xmn512m"
Modified bin / server.sh the JVM configuration, depending on the case may be produced, preferably not less than 1g
JAVA_OPT="${JAVA_OPT} -server -Xms1g -Xmx1g -Xmn512m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m"
Modify conf / 2m-noslave following broker configuration
broker-a.properties
# Belongs to a cluster name brokerClusterName = rocketmq- Cluster #broker name, pay attention here to fill in a different configuration file is not the same brokerName = broker- A document only difference between the two # # 0 for Master,> 0 = 0 represents brokerId Slave #nameServer address, separated by semicolons brokerId = 0 #nameServer address, separated by semicolons namesrvAddr = 10.32.16.195: 9876; 10.32.16.196: 9876 # sending the message, the server automatically creates a topic does not exist, the default number of queues created defaultTopicQueueNums =. 4 # whether Broker allows to automatically create Topic, turn on the advice of the line, the line is closed autoCreateTopicEnable = to true # allow Broker to automatically create a subscription group, the advice line open, online close autoCreateSubscriptionGroup = to true #Broker Foreign service listening port listenPort = 10911 # Delete the file time point, the default 4:00 deleteWhen = 04 # retention time files, the default 48 hours fileReservedTime = 120 #commitLog default size of each file. 1G mapedFileSizeCommitLog = 1073741824 #ConsumeQueue 30W each file in the default storage section, adjusted according to the service circumstances mapedFileSizeConsumeQueue = 300000 #destroyMapedFileIntervalForcibly = 120000 #redeleteHangedFileInterval = 120000 # detect a physical disk file space diskMaxUsedSpaceRatio = 88 # storage path storePathRootDir = / usr / local / rocketmq-4.5.2 / store #commitLog storage path storePathCommitLog = / usr / local / rocketmq- 4.5.2 / Store / commitlog # consumption queue storage path storage path storePathConsumeQueue = / usr / local / rocketmq-4.5.2 / Store / consumequeue # message index storage path storePathIndex = / usr / local / rocketmq-4.5.2 / Store / index #checkpoint file storage path storeCheckpoint = / usr / local / rocketmq -4.5.2 / store / the checkpoint #abort file storage path abortFile = / usr / local / rocketmq-4.5.2 / store / ABORT # message size limit MaxMessageSize = 65536 #flushCommitLogLeastPages =. 4 #flushConsumeQueueLeastPages = 2 #flushCommitLogThoroughInterval = 10000 # flushConsumeQueueThoroughInterval = 60000 #Broker roles # - ASYNC_MASTER asynchronous replication Master # - SYNC_MASTER Synchronous Dual Write Master # - SLAVE brokerRole = ASYNC_MASTER # brush disc mode # - ASYNC_FLUSH asynchronous brush disc # - SYNC_FLUSH synchronization brush disc flushDiskType = ASYNC_FLUSH #checkTransactionMessageEnable = to false number of thread pool # message #sendMessageThreadPoolNums = 128 # Pull message thread number pool #pullMessageThreadPoolNums = 128
broker-b.properties
# Belongs to a cluster name brokerClusterName = rocketmq- Cluster #broker name, pay attention here to fill in a different configuration file is not the same brokerName = broker- b # two documents only difference between the # 0 for Master,> 0 = 0 represents brokerId Slave #nameServer address, separated by semicolons brokerId = 0 #nameServer address, separated by semicolons namesrvAddr = 10.32.16.195: 9876; 10.32.16.196: 9876 # sending the message, the server automatically creates a topic does not exist, the default number of queues created defaultTopicQueueNums =. 4 # whether Broker allows to automatically create Topic, turn on the advice of the line, the line is closed autoCreateTopicEnable = to true # allow Broker to automatically create a subscription group, the advice line open, online close autoCreateSubscriptionGroup = to true #Broker Foreign service listening port listenPort = 10911 # Delete the file time point, the default 4:00 deleteWhen = 04 # retention time files, the default 48 hours fileReservedTime = 120 #commitLog default size of each file. 1G mapedFileSizeCommitLog = 1073741824 #ConsumeQueue 30W each file in the default storage section, adjusted according to the service circumstances mapedFileSizeConsumeQueue = 300000 #destroyMapedFileIntervalForcibly = 120000 #redeleteHangedFileInterval = 120000 # detect a physical disk file space diskMaxUsedSpaceRatio = 88 # storage path storePathRootDir = / usr / local / rocketmq-4.5.2 / store #commitLog storage path storePathCommitLog = / usr / local / rocketmq- 4.5.2 / Store / commitlog # consumption queue storage path storage path storePathConsumeQueue = / usr / local / rocketmq-4.5.2 / Store / consumequeue # message index storage path storePathIndex = / usr / local / rocketmq-4.5.2 / Store / index #checkpoint file storage path storeCheckpoint = / usr / local / rocketmq -4.5.2 / store / the checkpoint #abort file storage path abortFile = / usr / local / rocketmq-4.5.2 / store / ABORT # message size limit MaxMessageSize = 65536 #flushCommitLogLeastPages =. 4 #flushConsumeQueueLeastPages = 2 #flushCommitLogThoroughInterval = 10000 # flushConsumeQueueThoroughInterval = 60000 #Broker roles # - ASYNC_MASTER asynchronous replication Master # - SYNC_MASTER Synchronous Dual Write Master # - SLAVE brokerRole = ASYNC_MASTER # brush disc mode # - ASYNC_FLUSH asynchronous brush disc # - SYNC_FLUSH synchronization brush disc flushDiskType = ASYNC_FLUSH #checkTransactionMessageEnable = to false number of thread pool # message #sendMessageThreadPoolNums = 128 # Pull message thread number pool #pullMessageThreadPoolNums = 128
First start namesrv two servers
nohup sh mqnamesrv >/usr/local/rocketmq-4.5.2/logs/namesrv.log 2>&1 &
Broker restart the two servers
nohup sh mqbroker -c /usr/local/rocketmq-4.5.2/conf/2m-noslave/broker-a.properties >/dev/null 2>&1 &
View mq start situation: jps
The dual master mode started successfully.
Four, rocketmq console
Download open source project: https: //github.com/apache/rocketmq-externals
application.properties file, the project will be used to configure up rocketmq.config.namesrvAddr
打包：mvn clean package -Dmaven.test.skip=true
Run springboot project: java -jar xx.jar
During the service failed to start:
Firewall service is not closed due rocketmq
systemctl stop firewalld.service