RocketMQ
A, RocketMQ basic components
1, the basic components
Before looking at Rocket architecture, we first look at RocketMQ, each role are doing, what played a role.
I、NameServer
1, is mainly used to store the topic, the relationship between the broker functions like zookeeper.
2, between NameServer not communicate with each other, clustering support, does not affect other nodes after a hang up.
3, when the Producer and Consumer are to obtain information Broker, even if NameServer down, it will not affect the use, but not registered with the routing information, which is registered after NameServer down Producer, Consumer and Broker are not able to normal work.
4, NameServer stateless, NameServer stored Broker, information Topic is not persistent store, but dynamic and with the reported Broker of change, although NameServer their support persistence configuration, but not in general.
5, Broker sends a heartbeat packet to NameServer, will bring all Topic information, if the Topic of too much information, but the network is not very good condition but will lead to heart failure detection, NameServer will think Broker has hung out.
II、Broker
1, message container, receiving the push message provider and forwards the message corresponding to the specified consumer consumption, equivalent Broker RocketMQ processing logic of the server.
A、消息顺序写：在消息的写入过程中，所有的Topic共用一个文件，该文件大小为1G，如果文件大小没有满，是不会写入新的文件中的，这样能够保证顺序写盘，提高发消息的TPS；
B、消息跳跃读：MQ读取消息依赖系统PageCache，PageCache命中率越高，读性能越高，Linux平时也会尽量预读数据，使得应用直接访问磁盘的概率降低。
(See: https://blog.csdn.net/javahongxi/article/details/72956619)
2, load balancing and dynamic stretching
News is that there is in queue, you want to have a greater throughput can increase the number of queue, it can also increase the number of Broker node.
A、如果Topic的消息量比较大，但是该集群节点的压力并不是很大，可以增加该Topic下面的queue数量，Topic的队列数跟发送、消费速度成正比。
B、如果集群cpu的使用率已经到达85%以上，那么就需要通过增加集群节点来提高吞吐量，现有的节点数量已经无法满足业务需求了。
3, high availability, high reliability
A、高可用：搭建集群，一台服务器宕机并不会影响整体的服务。
B、高可靠：数据不允许丢失，有可靠的持久化机制。
4, heartbeat mechanism
Heartbeat mechanism is divided into two areas:
A、Broker每隔30s向NameServer发送心跳包，包含改Broker所有的Topic信息。
B、NameServer查询Broker心跳情况，如果某个Broker在2分钟内没有心跳，则认为该Broker已经宕机。
III、Producer
Message producer, generally at one end for initiating a service transmission MQ message.
1、Producer Group
Consumption of a class set is generally the same type of message sent, and send logically consistent.
2, the heartbeat mechanism Producer
A、Producer在启动的时候需要指定NameServer，启动后与NameServer保持长连接，并每隔30s从NameServer获取Topic的队列情况，获取到当前消费Topic和Broker的信息后，与相关的Broker保持长连接。
B、Producer每隔30s向Broker发送心跳包，同时Broker也会每隔10s扫描一次当前的Producer，如果2分钟没有发生心跳，那么认为该Producer已经移除，将会断开连接。
3, Producer message production
Sending a message type of the message is selected, the default will in turn sends a message to Producer Broker, to achieve load balancing purposes.
(Sent by the Producer knowledge can refer to this blog: https://blog.csdn.net/a1084986263/article/details/81223689)
IV、Consumer
Consumer news, general news consumption by the asynchronous backend systems.
1、Consumer Group
Producer Group similar concept.
2, Consumer heartbeat mechanism
Producer similar concept.
3, Consumer consumption patterns
A、MQPullConsumer：由用户控制线程，主动从服务端获取消息，需要自己维护offset，编程难度较大，不推荐使用。
B、MQPushConsumer：底层同样由pull实现，由用户注册的MessageListener来消费消息，不用自己维护offset。
2, the basic architecture (from RocketMQ official website)
In the figure we can see four roles can build clusters.
Producer and Consumer Needless to say this, the number of nodes can control our business systems, but can also be controlled to increase the number of members of the Group.
NameServer build a cluster when each node will have a complete information, so a NameServer downtime will not affect the whole system work, and in order to do disaster recovery, we can not NameServer server unified fabric in the region.
Broker Broker may be by increasing the number of guaranteed throughput of the system, while the master-slave mode may be used to increase system reliability, the specific configuration inside the package RocketMQ are exemplary.
Two, RocketMQ optimization
1, Producer optimization
1、一个应用尽可能只用一个 Topic，消息自类型可以用tags来标示，tags可以有应用自由设置，只需要producer和consumer使用一样的即可；
2、发送消息时，尽量设置keys，这样方便定位消息丢失的问题，例如可以使用订单id这样的主键作为消息的keys，因为keys是一种哈希索引，保证keys的唯一行可以在最大程度上避免哈希冲突。
3、如果消息的可靠性要求比较高，可以打印出消息日志进行追踪和定位。
4、如果消息属于同一个tag，同时消息的信息量又比较大，我们可以通过发送批量消息进行处理，提升性能。
5、建议消息大小不超过512K。
6、发送消息有同步和异步两种方式，如果对消息的性能要求比较高，可以使用异步的方式，因为同步发送的方式会阻塞。
7、send发送消息时，不抛异常，就代表发送成功。但是可以定义更加明确的返回状态。
8、对于消息不可丢失的业务，例如转账，必须使用消息重发机制。
Specific reference eventual consistency MQ distributed transaction to ensure consistency of approach and maintain a middle of the table to ensure the transaction.
2, Consumer Optimization
1、消费组和订阅
2, a message listener
public class Consumer {
public static final String NAME_SERVER_ADDR = "*.*.*.*:9876";
public static void main(String[] args) throws MQClientException {
// 1. 创建消费者（Push）对象
DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("GROUP_TEST");
// 2. 设置NameServer的地址，如果设置了环境变量NAMESRV_ADDR，可以省略此步
consumer.setNamesrvAddr(NAME_SERVER_ADDR);
// 消费重试次数 -1代表16次
consumer.setMaxReconsumeTimes(-1);
// 3. 订阅对应的主题和Tag
consumer.subscribe("TopicTest", "*");
// 4. 注册消息接收到Broker消息后的处理接口
consumer.registerMessageListener(new MessageListenerConcurrently() {
public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> list, ConsumeConcurrentlyContext consumeConcurrentlyContext) {
try {
MessageExt messageExt = list.get(0);
System.out.printf("线程：%-25s 接收到新消息 %s --- %s %n", Thread.currentThread().getName(), messageExt.getTags(), new String(messageExt.getBody(), RemotingHelper.DEFAULT_CHARSET));
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
}
return ConsumeConcurrentlyStatus.RECONSUME_LATER;
}
});
// 5. 启动消费者(必须在注册完消息监听器后启动，否则会报错)
consumer.start();
System.out.println("已启动消费者");
}
}
当消息消费失败时，消息的存储方式将会发生改变，当前的消息会变为一个延时消息，默认等级为3，延时时间为10s，并且随着消费失败的次数提升延时消息的等级，设置-1的时候默认尝试重新消费16次。
Non-sequential message, if slow process, a thread can re-open their own, do not block the queue, but if the order message, open a thread can not solve the blocking problem.
3, JVM and Linux configuration
1、JVM配置
2、Linux配置
If you do not know Linux configuration and principle, it is best to keep the default parameters.
