RocketMQ-Core (1)-Software Installation

RocketMQ series description

This series mainly updates the core foundation of RocketMQ & SringBoot integration & source code analysis. All the content of the article is a personal test practice. If there are mistakes, you are welcome to criticize and correct

Reference article description: In addition to the official website documents, if there are friends who need information and books, please leave an email and send them to everyone in time

• Still Silicon Valley Distributed Message Queue RocketMQ Notes
• RocketMQ actual combat and principle analysis-Yang Kaiyuan
• Official website document

This section mainly talks about installation. You can also read the concept introduction section later and then come back to this section.

1. Download RocketMQ

1. Official website address: http://rocketmq.apache.org/
2. Download address: http://rocketmq.apache.org/release_notes/release-notes-4.9.1/
3. Source address: https://github.com/apache/rocketmq
4. Chinese documentation: https://github.com/apache/rocketmq/tree/master/docs/cn
5. On the download page, download version 4.9.1 of RocketMQ
6. After the download is complete, upload to the corresponding server (using CentOS7)

2. RocketMQ installation

1. Unzip the compressed package

unzip rocketmq-all-4.9.1-bin-release.zip

2. The name is too long, you can choose whether to repeat the name

mv rocketmq-all-4.9.1-bin-release rocketmq-4.9.1

3. Directory Structure
   • benchmark : includes a shell script that runs the benchmark program
   • bin : Contains various RocketMQ shell scripts (Linux platform) and cmd scripts (Windows platform), such as the commonly used script mqnamesrv to start NameServer , the script mqbroker to start Broker, the cluster management script mqadmin, etc.
   • conf : Sample configuration files, including three types of broker files, logback files, etc. When users configure files, they generally base on these sample configuration files and add their own special needs
   • lib : Including the jar packages compiled by each module of RocketMQ and the jar packages that RocketMQ depends on

[root@server01 rocketmq-4.9.1]# ll
total 40
drwxr-xr-x. 2 root root   126 Oct 24 14:24 benchmark
drwxr-xr-x. 3 root root  4096 Oct 24 14:24 bin
drwxr-xr-x. 6 root root   211 Oct 24 14:24 conf
drwxr-xr-x. 2 root root  4096 Oct 24 14:24 lib
-rwxr-xr-x. 1 root root 17327 Oct 24 14:24 LICENSE
-rwxr-xr-x. 1 root root  1338 Oct 24 14:24 NOTICE
-rwxr-xr-x. 1 root root  5132 Oct 24 14:24 README.md

4. Therefore, the installation of RocketMQ only needs to be decompressed to complete the installation
5. The linux installation is basically the same as the windows installation, just unzip it, but the startup script is different

3. RocketMQ startup

3.1 Linux

1. It is relatively simple to start the message queue service of a single machine, no need to write configuration files, just start the NameServer and Broker of the machine in turn
2. Start NameServer

sh bin/mqnamesrv &

• command description

usage: mqnamesrv [-c <arg>] [-h] [-n <arg>] [-p]
 -c,--configFile <arg>    Name server config properties file
 -h,--help                Print help
 -n,--namesrvAddr <arg>   Name server address list, eg: 192.168.0.1:9876;192.168.0.2:9876
 -p,--printConfigItem     Print all config item

• Log directory: tail -f ~/logs/rocketmqlogs/namesrv.log

3. Start Brocker

sh bin/mqbroker -n 127.0.0.1:9876 -c conf/broker.conf &

• command description

usage: mqbroker [-c <arg>] [-h] [-m] [-n <arg>] [-p]
 -c,--configFile <arg>       Broker config properties file
 -h,--help                   Print help
 -m,--printImportantConfig   Print important config item
 -n,--namesrvAddr <arg>      Name server address list, eg: 192.168.0.1:9876;192.168.0.2:9876
 -p,--printConfigItem        Print all config item

• If it is a public network, you need to specify the NameServer address as the public network address
  • No need to specify http/https protocol

sh bin/mqbroker -n www.xxxx.com:9876 -c conf/broker.conf &

• Log path: tail -f ~/logs/rocketmqlogs/broker.log

4. Close broker and namesrv

> sh bin/mqshutdown broker
The mqbroker(36695) is running...
Send shutdown request to mqbroker(36695) OK

> sh bin/mqshutdown namesrv
The mqnamesrv(36664) is running...
Send shutdown request to mqnamesrv(36664) OK

5. In order to avoid excessive memory usage, you can modify the runserver.sh and runbroker.sh files to specify the memory size
6. After the startup is complete, you can view the progress through jps

[root@TianXinCoord rocketmq_source_code_topic]# jps
4241 BrokerStartup # broker
4214 NamesrvStartup # 名称服务
9708 Jps

3.2 Windows

1. Add environment variables
   • D:\rocketmq\server: store decompressed RocketMQ files

ROCKETMQ_HOME="D:\rocketmq\server"
NAMESRV_ADDR="127.0.0.1:9876"

2. Start NameServer

.\bin\mqnamesrv.cmd

Java HotSpot(TM) 64-Bit Server VM warning: Using the DefNew young collector with the CMS collector is deprecated and will likely be removed in a future release
Java HotSpot(TM) 64-Bit Server VM warning: UseCMSCompactAtFullCollection is deprecated and will likely be removed in a future release.
The Name Server boot success. serializeType=JSON

3. Start Broker
   • autoCreateTopicEnable=true: Enable automatic topic creation, if it is turned off, it needs to be created manually

./bin/mqbroker -n 127.0.0.1:9876 -c conf/broker.conf

The broker[LAPTOP-IA8CO8JN, 172.31.240.22:10911] boot success. serializeType=JSON and name server is rocketmq01.com:9876

4. To close the corresponding service, just close the window

4. RocketMQ Console

1. Download source code: https://github.com/apache/rocketmq-externals/tree/release-rocketmq-console-1.0.0
   • Pay attention to the branch information: release-rocketmq-console-1.0.0 , the console is one of the branches of the rocketmq-externals project
2. Find rocketmq-console, edit the application.properties configuration file, and configure the rocketmq.config.namesrvAddr used by the project (or configure it in the form of parameters when the project starts)
   • server.port=8888
   • rocketmq.config.namesrvAddr=127.0.0.1:9876
     • Multiple NameServers are separated by commas

server.contextPath=
server.port=8888
#spring.application.index=true
spring.application.name=rocketmq-console
spring.http.encoding.charset=UTF-8
spring.http.encoding.enabled=true
spring.http.encoding.force=true
logging.config=classpath:logback.xml
#if this value is empty,use env value rocketmq.config.namesrvAddr  NAMESRV_ADDR | now, you can set it in ops page.default localhost:9876
rocketmq.config.namesrvAddr=localhost:9876
#if you use rocketmq version < 3.5.8, rocketmq.config.isVIPChannel should be false.default true
rocketmq.config.isVIPChannel=
#rocketmq-console's data path:dashboard/monitor
rocketmq.config.dataPath=/tmp/rocketmq-console/data
#set it false if you don't want use dashboard.default true
rocketmq.config.enableDashBoardCollect=true

3. Pack rocktmq-console into a jar package to get rocketmq-console-ng-1.0.0.jar

mvn clean package -Dmaven.test.skip=true

4. Run the jar package, start the project, here you can also set rocketmq.config.namesrvAdd

java -jar rocketmq-console-ng-1.0.0.jar --server.port=8888 --rocketmq.config.namesrvAddr=127.0.0.1:9876

5. Just visit the address: http://localhost:8888

5. Security configuration

5.1 Public network access

1. When RocketMQ is not configured, it can only be local by default. If external network access is required, it needs to be reconfigured
2. Modify conf/broker.conf to specify the brokerIp of the name service address
   • When the machine has multiple network cards, specify through brokerIP1=<specified id>, otherwise the wrong address may be identified

brokerClusterName = DefaultCluster
brokerName = broker-a
brokerId = 0
deleteWhen = 04
fileReservedTime = 48
namesrvAddr=公网IP:9876
brokerIP1=公网IP
brokerRole = ASYNC_MASTER
flushDiskType = ASYNC_FLUSH

3. Specify the configuration file when starting the broker

sh bin/mqbroker -n <公网ip/域名>:9876 -c conf/broker.conf &

4. Console access also needs to configure the corresponding name service

java -jar rocketmq-console-ng-1.0.0.jar --server.port=8888 --rocketmq.config.namesrvAddr=<公网ip/域名>:9876

5.2 Modify port

1. namesrv listens on port 9876 by default, and you can also specify a custom port
2. Create a new property configuration file, for example, create a new namesrv.propertes configuration under /rocketmq/config/, specify listenPort in it

listenPort=9876

3. When starting namesrv, specify the location of the configuration file by -c <path>