RocketMQ source code analysis——Producer

Message sending code implementation

The following is a demo of a producer sending messages (synchronous sending)

Mainly done several things:

• Initialize a producer (DefaultMQProducer) object
• Set the address of NameServer
• Start producer
• Send a message

The message sender initiates the process

DefaultMQProducerImpl类start()

Check configuration

DefaultMQProducerImpl类

Get an MQ client instance

There is only one MQClientManager instance in the entire JVM, and an MQClientInstance cache table is maintained. DefaultMQProducerImpl class start()

A clientId will only create one MQClientInstance

clientId generation rule: IP@instanceName@unitName

In RocketMQ, message senders and message consumers are both "clients". Each client is an MQClientInstance, and each ClientConfig corresponds to an instance.

If different producers and consumers reference the same client configuration (ClientConfig), they share an MQClientInstance instance. So we should pay attention to this problem when defining it (if the group name of the producer and consumer is the same, it will easily cause this problem)

Start an instance

MQClientInstance class start()

scheduled tasks

MQClientInstance类startScheduledTask()

Producer message sending process

We can see from the code of a producer case: sendDefaultImpl() in DefaultMQProducerImpl is the core method for sending producer messages.

From the core method, we can see that message sending consists of four steps: verifying the message, finding the route, selecting the queue, and sending the message.

Select queue

Queue policy selected by default

The simplest polling algorithm is adopted. This algorithm has a good feature that it ensures that the number of message delivery in each Queue queue is as even as possible. This algorithm can basically ensure that the number of message delivery in each Queue queue is as even as possible as long as no retries occur during the message delivery process. Of course, if a problem occurs during delivery, for example, the first delivery fails, then it is very likely that a Broker in the cluster state is down, so avoid it by retrying the delivery. This setting is also more reasonable.

Failure delay mechanism strategy*

After adopting this strategy, RocketMQ will calculate the available time of the Borker (send end time - send start time, failure will be calculated as 30S) every time it is successfully or abnormally sent to the Broker, and save it to facilitate filtering when sending next time. .

In addition to recording the length of time the Broker sends messages, it is also necessary to calculate the length of time a Broker is unavailable. An empirical value is used here:

If the sending time is within 550ms, the unavailable time is 0.

Reach 550ms, unavailable for 30S

Reaching 1000ms, the unavailable time is 60S

Reaching 2000ms, the unavailable time is 120S

Reaching 3000ms, the unavailable time is 180S

Reached 15000ms, unavailable for 600S

With the above Broker evasion information, sending messages is very simple.

The steps to enable the fault delay mechanism strategy are as follows:

1. Carry out rotation training according to the message queue list
2. Choose a queue
3. Determine whether the Broker where the queue is located is available
4. If it is available, the queue is returned and the queue selection logic ends.
5. If not available, continue with step 2
6. If neither is available, choose one at random

code show as below:

Choice of two strategies

From this strategy, it can be clearly seen that the default queue selection is a polling strategy, while the failure delay selection queue is a queue that gives priority to the message sending time. So how to choose?

First of all, RocketMQ has a retry strategy for default sending failure. The default is 2. That is, if sending to different Brokers fails three times, then the sending of this message will fail. As RocketMQ, it must try its best to ensure that the message is sent successfully. So the following suggestions are given.

If the network is relatively good, the default strategy is recommended. After all, the probability of sending failure caused by network problems is relatively small.

If the network is not very good, a fault delay mechanism is recommended. When selecting a message queue, it will filter out brokers that RocketMQ considers to be unavailable within a period of time to avoid continuously sending messages to downed brokers, thereby enabling message sending. High availability.

Of course, the condition for the above to be true is that a Topic is created on the basis of two or more Brokers.

Technology Highlight: ThreadLocal