Earlier we briefly introduced the commonly used message middleware. In this article, we mainly introduce the use of the message middleware RocketMQ producer. RocketMQ implements two types of
producers, DefaultMQProducer and TransactionMQProducer, the former produces ordinary messages and the latter produces transaction messages. In this article, we mainly introduce the use of DefaultMQProducer. Before reading this article, you need to understand the overall structure of RocketMQ and the related concepts of RocketMQ. These contents have been written very clearly on the RocketMQ official website. If you are unclear, you can refer to RocketMQ GitHub .
Before that, we first introduce the use of Message. Message is RocketMQ's encapsulation of messages. We can only encapsulate messages as Message instances before they can be sent out through RocketMQ. First, let's take a look at RocketMQ's definition of messages:
public class Message implements Serializable {
private static final long serialVersionUID = 8445773977080406428L;
//主题可以通过RocketMQ Console创建
private String topic;
//消息扩展信息，Tag、keys、消息延迟级别都保存在Map中
private Map<String, String> properties;
//目前没用
private int flag;
//消息体，字节数组
private byte[] body;
//事务ID
private String transactionId;
//设置消息的key，多个Key可以用MessageConst.KEY_SEPARATOR分隔或者直接调用第二个方法，传入一个集合
//最终保存在 properties中， key为MessageConst.PROPERTY_KEYS
public void setKeys(String keys) {}
public void setKeys(Collection<String> keys) { }
//设置主题，也可以通过构造传入
public void setTopic(String topic) {}
//设置Tag，消息过滤标记，用户可以订阅topic的某些Tag
public void setTags(String tags) {}
//设置延迟级别，延迟多久消费者可以消费
public void setDelayTimeLevel(int level) {}
//设置消息体，可通过构造传入
public void setBody(byte[] body) {}
//设置是否等消息存储成功
public void setWaitStoreMsgOK(boolean waitStoreMsgOK) {}
//设置用户ID
public void setBuyerId(String buyerId) {}
//设置事务ID
public void setTransactionId(String transactionId) {}
//设置属性properties key为name value为value
void putProperty(final String name, final String value) {}
//放置其他扩展信息，最终调用putProperty(final String name, final String value)
public void putUserProperty(final String name, final String value) {}
//设置properties
void setProperties(Map<String, String> properties) { }
//设置flag
public void setFlag(int flag) {}
}After learning the structure of Message, we started to learn how DefaultMQProducer sends messages. DefaultMQProducer provides multiple APIs for sending different types of messages. The following is the message API provided by DefaultMQProducer. The exception of the method and the main logic are not shown here, you can refer to the source code.
/*以下发送方法以同步的模式发送消息，该方法只有整个发送流程全部完成之后，才会返回。该方法有内部
* 的重试机制，可以通过参数retryTimesWhenSendFailed控制，因此可能有多个消息发送到broker，应
* 应用程序开发
* 人员需要解决潜在的重复问题
* 第二个方法额外指定了超时时间
* 第三个方法额外指定了队列MessageQueue，后续介绍
* 第四个额外指定了超时时间和队列MessageQueue
* 第五个方法额外指定了消息选择器MessageQueueSelector，后续介绍
* 第六个额外指定了超时时间和消息选择器MessageQueueSelector
**/
public SendResult send(Message msg) {}
public SendResult send(Message msg, long timeout) {}
public SendResult send(Message msg, MessageQueue mq){}
public SendResult send(Message msg, MessageQueue mq, long timeout){}
public SendResult send(Message msg, MessageQueueSelector selector, Object arg){}
public SendResult send(Message msg, MessageQueueSelector selector, Object arg, long timeout){}
/*
* 以下方法以异步的模式发送消息到broker，该方法将会直接返回，一旦发送流程完成，会执行
* sendCallback回调与send(Message)类似，它内部实现也会在重试次数用完之后失败才会声明发送失败，
* 因此也需要应用程序开发人员解决消息重复的问题
* 其他方法的参数，比如timeout,MessageQueue,MessageQueueSelector 与上面介绍的一样
**/
public void send(Message msg,SendCallback sendCallback) {}
public void send(Message msg, SendCallback sendCallback, long timeout){}
public void send(Message msg, MessageQueue mq, SendCallback sendCallback){}
public void send(Message msg, MessageQueue mq, SendCallback sendCallback, long timeout){}
public void send(Message msg, MessageQueueSelector selector, Object arg, SendCallback sendCallback){}
public void send(Message msg, MessageQueueSelector selector, Object arg, SendCallback sendCallback, long timeout){}
/*
* 以下类似于UDP报文协议，此方法发送消息之后不会等待broker确认，也就是不关注broker是否收到消息
* 它能最大程度提高吞吐量，但潜在会有消息丢失。其他参数与前面介绍的一致
/**
public void sendOneway(Message msg){}
public void sendOneway(Message msg, MessageQueue mq) {}
public void sendOneway(Message msg, MessageQueueSelector selector, Object arg){}
/*
* 以下方法用于批量发送消息其他参数与前面介绍的一致
**/
public SendResult send(Collection<Message> msgs){}
public SendResult send(Collection<Message> msgs, long timeout){}
public SendResult send(Collection<Message> msgs, MessageQueue messageQueue) {}
public SendResult send(Collection<Message> msgs, MessageQueue messageQueue, long timeout){}Above we introduced the API for DefaultMQProducer to send messages. Below we use examples to see how the DefaultMQProducer is used. We only write one or two of the above sending modes. In general, the producer's message sending logic is relatively simple, just connect to the RocketMQ service, and then call the send method, the code is as follows:
//使用生产者组名实例化一个生产者
DefaultMQProducer producer = new DefaultMQProducer("DefaultProducer");
// 指定RocketMQ nameServer地址
producer.setNamesrvAddr("10.0.10.63:9876");
// 启动生产者
producer.start();
//创建Message实例
Message msg = new Message("BenchmarkTest" , "TagA", ("Hello RocketMQ").getBytes(RemotingHelper.DEFAULT_CHARSET)
//调用sendOneway()发送消息，该方法不会管消息是否发送成功
producer.sendOneway(msg);
//同步发送消息，根据sendResult结果处理
SendResult sendResult = producer.send(msg);
//异步发送消息
producer.send(msg, new SendCallback() {
public void onSuccess(SendResult sendResult) {
//发送成功,业务处理
}
public void onException(Throwable e) {
//发送异常，业务处理
}
});Previously introduced the use of DefaultMQProducer to send synchronous and asynchronous messages. The above example did not involve MessageQueue and MessageQueueSelector. Below we briefly introduce the use of these two, first we introduce the use of MessageQueueSelector. Before introducing MessageQueueSelector, let's take a look at the process of RocketMQ sending messages.
In RocketMQ, Topic is a logical concept that represents the same type of message, and the message will eventually be sent to the queue. MessageQueue and MessageQueueSelector allow us to send messages to the specified queue according to certain rules. MessageQueueSelector has three strategies, namely SelectMessageQueueByRandom, SelectMessageQueueByHash, and SelectMessageQueueByMachineRoom. The algorithm of SelectMessageQueueByHash is to take the absolute value of the hashcode of the arg parameter, and then take the remainder of mqs.size() to get the subscript
SelectMessageQueueByRandom of the target queue in mqs. Its algorithm is to directly use a random value as the target queue in mqs according to mqs.size() The
method of SelectMessageQueueByMachineRoom currently returns null. The following is an example of MessageQueueSelector:
MessageQueueSelector selector = new SelectMessageQueueByHash();
SendResult sendResult = producer.send(msg, selector, 105);MessageQueueSelector specifies a certain rule. RocketMQ selects a queue according to the rule, while MessageQueue directly specifies a fixed queue, as shown in the following example:
MessageQueue queue = new MessageQueue();
queue.setBrokerName("sss");
queue.setQueueId(2);
queue.setTopic("queueTpoic");
SendResult sendResult = producer.send(msg,queue);Earlier we introduced the API for DefaultMQProducer to send messages, as well as the specific parameters of MessageQueue, MessageQueueSelector, and DefaultMQProducer. When we created DefaultMQProducer, we only set the address of the nameServer. Below we introduce the core properties of DefaultMQProducer. The core properties are as follows:
//设置nameSrvAddr地址，如果有多个则以分号隔开。
public void setNamesrvAddr(String namesrvAddr) {}
//设置客户端所在的IP
public void setClientIP(String clientIP) {}
//设置实例名称，每个实例需要取唯一的名字。
public void setInstanceName(String instanceName) {}
//表示是否开启VIP通道，VIP和非VIP区别是使用的端口不同
public void setVipChannelEnabled(final boolean vipChannelEnabled) {}
//客户端回调线程数，表示Netty通信层回调线程的个数
public void setClientCallbackExecutorThreads(int clientCallbackExecutorThreads) {}
//获取Topic路由信息的间隔时长，单位为毫秒，默认为30000毫秒
public void setPollNameServerInterval(int pollNameServerInterval) {}
//与Broker心跳间隔的时长，单位为毫秒，默认为30000毫秒
public void setHeartbeatBrokerInterval(int heartbeatBrokerInterval) {}
//生产者组这是一个必须传递的参数同一个生产者组中的生产者实例行为需要一致。
public void setProducerGroup(String producerGroup) {}
//发送超时时间 单位为毫秒
public void setSendMsgTimeout(int sendMsgTimeout) {}
//消息的容量上限，超时该值会通过ZIP压缩，默认为4M
public void setCompressMsgBodyOverHowmuch(int compressMsgBodyOverHowmuch) {}
//同步发送消息失败重试的次数，默认两次
public void setRetryTimesWhenSendFailed(int retryTimesWhenSendFailed){}
//异步发送消息失败重试的次数，默认两次
public void setRetryTimesWhenSendAsyncFailed(final int retryTimesWhenSendAsyncFailed){}
//在发送消息时，自动创建服务器不存在的topic，需要指定Key，该Key可用于配置发送消息所在topic的默认路由。
public void setCreateTopicKey(String createTopicKey){}
//在发送消息，自动创建服务器不存在的topic时，默认创建的队列数
public void setDefaultTopicQueueNums(int defaultTopicQueueNums){}
//如果发送消息返回sendResult，但是sendStatus!=SEND_OK，是否重试发送另一个broker 默认为false
public void setRetryAnotherBrokerWhenNotStoreOK(boolean retryAnotherBrokerWhenNotStoreOK) {}The above are the core parameters of DefaultMQProducer. In addition, we can also pass in an instance of RPCHooK when instantiating DefaultMQProducer. The RPCHooK instance includes pre-processing before sending the message and processing after the message response. Users can use the first interface Do some security control or other operations. The following is the definition of the RPCHooK interface and the construction definition of DefaultMQProducer:
public interface RPCHook {
void doBeforeRequest(final String remoteAddr, final RemotingCommand request);
void doAfterResponse(final String remoteAddr, final RemotingCommand request, final RemotingCommand response);
}
public DefaultMQProducer(final String producerGroup, RPCHook rpcHook) {
this.producerGroup = producerGroup;
defaultMQProducerImpl = new DefaultMQProducerImpl(this, rpcHook);
}Except for sendOneway (Message message), the result of sending the message will be returned regardless of whether the message is sent synchronously or asynchronously, and the result is encapsulated in the SendResult instance. The following is the definition of SendResult:
public class SendResult {
private SendStatus sendStatus;
private String msgId;
private MessageQueue messageQueue;
private long queueOffset;
private String transactionId;
private String offsetMsgId;
private String regionId;
private boolean traceOn = true;
}In the above field, the SendStatus we are going to talk about here represents the status of sending the message. As long as the send message method does not throw an exception, it means the sending is successful. There will be multiple statuses when sending successfully, the definition of SendStatus is as follows:
public enum SendStatus {
SEND_OK,
FLUSH_DISK_TIMEOUT,
FLUSH_SLAVE_TIMEOUT,
SLAVE_NOT_AVAILABLE,
}- SEND_OK: The message was sent successfully. It should be noted that the success of the message does not mean that it is reliable. To ensure that no messages will be lost, you should also enable synchronous Master server or synchronous flashing, namely SYNC_MASTER or SYNC_FLUSH.
- FLUSH_DISK_TIMEOUT: The message is sent successfully but the server flashing timeout. At this point, the message has entered the server queue (memory), and the message will only be lost if the server is down. The message storage configuration parameters can set the flashing mode and the length of time for synchronous flashing. If the Broker server has set the flashing mode to be synchronous flashing, that is, FlushDiskType=SYNC_FLUSH (default is asynchronous flashing mode), when the Broker server is not flashing synchronously If the flashing is completed within the flashing time (the default is 5s), it will return to this state-flashing timeout.
- FLUSH_SLAVE_TIMEOUT: The message was sent successfully, but the server timed out when synchronizing to the Slave. At this point, the message has entered the server queue, and the message will be lost only if the server is down. If the role of the Broker server is the synchronous Master, that is, SYNC_MASTER (the default is the asynchronous Master or ASYNC_MASTER), and the slave Broker server does not complete the synchronization with the master server within the synchronization flashing time (default is 5 seconds), it will return to this state— —Data synchronization to the Slave server timed out.
- SLAVE_NOT_AVAILABLE: The message was sent successfully, but the slave is not available at this time. If the role of the Broker server is the synchronous Master, that is, SYNC_MASTER (the default is the asynchronous Master server, that is, ASYNC_MASTER), but there is no slave Broker server configured, it will return to this state-no slave server is available.
This article introduces the use of RockerMQ producer DefaultMQProducer in detail. Relatively speaking, the use of DefaultMQProducer is relatively simple, mainly the use of Send method. In the next article, we will introduce the use of RockerMQ consumers. As for RocketMQ, RocketMQ deployment and other issues, you can check the RocketMQ official website by yourself .