Open message track
broker side
traceTopicEnableThe property is set to true, and the default value is false. Set to true, the default topic for storing trajectory data will be initialized when the broker starts: RMQ_SYS_TRACE_TOPIC;The
traceOnattribute is set to true, and the default value is also true. If this property is set to false, the client will not send trace data to the broker
producer
When constructing the producer object, set enableMsgTrace=true, customizedTraceTopic can be empty, use the default topic, and other overloaded interfaces are similar
/**
* Constructor specifying producer group, enabled msgTrace flag and customized trace topic name.
*
* @param producerGroup Producer group, see the name-sake field.
* @param enableMsgTrace Switch flag instance for message trace.
* @param customizedTraceTopic The name value of message trace topic.If you don't config,you can use the default
* trace topic name.
*/
public DefaultMQProducer(final String producerGroup, boolean enableMsgTrace, final String customizedTraceTopic) {
this(null, producerGroup, null, enableMsgTrace, customizedTraceTopic);
}
consumer
When constructing the consumer object, set enableMsgTrace=true, customizedTraceTopic can be empty, use the default topic, and other overloaded interfaces are similar
/**
* Constructor specifying consumer group, enabled msg trace flag and customized trace topic name.
*
* @param consumerGroup Consumer group.
* @param enableMsgTrace Switch flag instance for message trace.
* @param customizedTraceTopic The name value of message trace topic.If you don't config,you can use the default trace topic name.
*/
public DefaultMQPushConsumer(final String consumerGroup, boolean enableMsgTrace, final String customizedTraceTopic) {
this(null, consumerGroup, null, new AllocateMessageQueueAveragely(), enableMsgTrace, customizedTraceTopic);
}
Storage medium for message trace data
The message trajectory data is still stored in RocketMQ's broker, and each trajectory data is sent to the specified topic just like a normal message.
The ID and KEYS of the original message will be used as the KEYS of the trajectory message, which can be used to retrieve the trajectory data of the specified message.
One advantage of not using external storage media is to avoid relying on third-party components.
How Producer collects trajectory data
Register a SendMessageHook when initializing the producer, collect the context information of the message before and after the message is sent, and asynchronously deliver the trajectory data to the broker after the message is sent.
@Override
public void sendMessageBefore(SendMessageContext context) {
//if it is message trace data,then it doesn't recorded
if (context == null || context.getMessage().getTopic().startsWith(((AsyncTraceDispatcher) localDispatcher).getTraceTopicName())) {
return;
}
//build the context content of TuxeTraceContext
TraceContext tuxeContext = new TraceContext();
tuxeContext.setTraceBeans(new ArrayList<TraceBean>(1));
context.setMqTraceContext(tuxeContext);
tuxeContext.setTraceType(TraceType.Pub);
tuxeContext.setGroupName(NamespaceUtil.withoutNamespace(context.getProducerGroup()));
//build the data bean object of message trace
TraceBean traceBean = new TraceBean();
// 发送前采集的轨迹数据如下
traceBean.setTopic(NamespaceUtil.withoutNamespace(context.getMessage().getTopic()));
traceBean.setTags(context.getMessage().getTags());
traceBean.setKeys(context.getMessage().getKeys());
traceBean.setStoreHost(context.getBrokerAddr());
traceBean.setBodyLength(context.getMessage().getBody().length);
traceBean.setMsgType(context.getMsgType());
traceBean.setClientHost(((AsyncTraceDispatcher)localDispatcher).getHostProducer().getmQClientFactory().getClientId());
tuxeContext.getTraceBeans().add(traceBean);
// 发送前采集部分数据到上下文
}
@Override
public void sendMessageAfter(SendMessageContext context) {
//if it is message trace data,then it doesn't recorded
if (context == null || context.getMessage().getTopic().startsWith(((AsyncTraceDispatcher) localDispatcher).getTraceTopicName())
|| context.getMqTraceContext() == null) {
return;
}
if (context.getSendResult() == null) {
return;
}
if (context.getSendResult().getRegionId() == null
|| !context.getSendResult().isTraceOn()) {
// if switch is false,skip it
return;
}
TraceContext tuxeContext = (TraceContext) context.getMqTraceContext();
// traceBean里保存了发送前采集的相关信息
TraceBean traceBean = tuxeContext.getTraceBeans().get(0);
// 发送耗时，traceBeans实际只会有一条数据
int costTime = (int) ((System.currentTimeMillis() - tuxeContext.getTimeStamp()) / tuxeContext.getTraceBeans().size());
tuxeContext.setCostTime(costTime);
if (context.getSendResult().getSendStatus().equals(SendStatus.SEND_OK)) {
tuxeContext.setSuccess(true);
} else {
tuxeContext.setSuccess(false);
}
tuxeContext.setRegionId(context.getSendResult().getRegionId());
traceBean.setMsgId(context.getSendResult().getMsgId());
traceBean.setOffsetMsgId(context.getSendResult().getOffsetMsgId());
// 计算存储时间方式：就是认为总耗时的一半，所以这不一个准确值
traceBean.setStoreTime(tuxeContext.getTimeStamp() + costTime / 2);
// 准备异步发送轨迹数据，并不是立即发送
localDispatcher.append(tuxeContext);
}
How the Consumer collects trajectory data
The consusmer is similar to the producer. It registers a ConsumeMessageHook, but the biggest difference from the producer is that the producer sends the track data of the message after the message is sent, but the consumer collects some data and sends it once before consumption, and then collects some data and sends it after consumption. At one time, consumption is a total of 2 trajectory data. If the consumption fails to retry, you will record two more for each retry.
@Override
public void consumeMessageBefore(ConsumeMessageContext context) {
if (context == null || context.getMsgList() == null || context.getMsgList().isEmpty()) {
return;
}
TraceContext traceContext = new TraceContext();
context.setMqTraceContext(traceContext);
traceContext.setTraceType(TraceType.SubBefore);//
traceContext.setGroupName(NamespaceUtil.withoutNamespace(context.getConsumerGroup()));//
List<TraceBean> beans = new ArrayList<TraceBean>();
for (MessageExt msg : context.getMsgList()) {
if (msg == null) {
continue;
}
String regionId = msg.getProperty(MessageConst.PROPERTY_MSG_REGION);
String traceOn = msg.getProperty(MessageConst.PROPERTY_TRACE_SWITCH);
if (traceOn != null && traceOn.equals("false")) {
// If trace switch is false ,skip it
continue;
}
TraceBean traceBean = new TraceBean();
traceBean.setTopic(NamespaceUtil.withoutNamespace(msg.getTopic()));//
traceBean.setMsgId(msg.getMsgId());//
traceBean.setTags(msg.getTags());//
traceBean.setKeys(msg.getKeys());//
traceBean.setStoreTime(msg.getStoreTimestamp());//
traceBean.setBodyLength(msg.getStoreSize());//
traceBean.setRetryTimes(msg.getReconsumeTimes());//
traceBean.setClientHost(((AsyncTraceDispatcher)localDispatcher).getHostConsumer().getmQClientFactory().getClientId());
traceContext.setRegionId(regionId);//
beans.add(traceBean);
}
if (beans.size() > 0) {
traceContext.setTraceBeans(beans);
traceContext.setTimeStamp(System.currentTimeMillis());
localDispatcher.append(traceContext);//消费前发送一次
}
}
@Override
public void consumeMessageAfter(ConsumeMessageContext context) {
if (context == null || context.getMsgList() == null || context.getMsgList().isEmpty()) {
return;
}
TraceContext subBeforeContext = (TraceContext) context.getMqTraceContext();
if (subBeforeContext.getTraceBeans() == null || subBeforeContext.getTraceBeans().size() < 1) {
// If subbefore bean is null ,skip it
return;
}
TraceContext subAfterContext = new TraceContext();
subAfterContext.setTraceType(TraceType.SubAfter);//
subAfterContext.setRegionId(subBeforeContext.getRegionId());//
subAfterContext.setGroupName(NamespaceUtil.withoutNamespace(subBeforeContext.getGroupName()));//
subAfterContext.setRequestId(subBeforeContext.getRequestId());//
subAfterContext.setSuccess(context.isSuccess());//
// Caculate the cost time for processing messages
int costTime = (int) ((System.currentTimeMillis() - subBeforeContext.getTimeStamp()) / context.getMsgList().size());
subAfterContext.setCostTime(costTime);//
subAfterContext.setTraceBeans(subBeforeContext.getTraceBeans());
String contextType = context.getProps().get(MixAll.CONSUME_CONTEXT_TYPE);
if (contextType != null) {
subAfterContext.setContextCode(ConsumeReturnType.valueOf(contextType).ordinal());
}
localDispatcher.append(subAfterContext);//消费后发送一次
}
How to send trajectory data
When the client sends or consumes a message, putting the trace message into a blocking queue is over. An asynchronous thread will take the trace message from this queue and encapsulate it as a sending task, submit it to the thread pool, and then send it to the broker.
- The default size of the queue for storing pending trace messages is 1024. If it is full, the current trace messages will be discarded after logging.
- There is an asynchronous thread that continuously polls and retrieves data from the queue of stored trajectory messages, up to 100 pieces each time (or waiting for 5ms is not enough 100), encapsulated as a sending request task, and submitted to the thread pool that sends the trajectory message
- The sending task categorizes this batch of messages by topic, and processes a batch of messages per topic and sends them to the trajectory topic in batches. The message ID and message keys of the original message are used as the keys of the trajectory message and the meta of this batch of original messages. Data (if there are more than one, the metadata of each message is finally spliced into one, and the metadata of each message has a field separator at the end, which can be used to split when querying) as the message body
How to query trajectory data
Because the message trajectory data is sent to the specified trajectory topic, the ID of the original message and the message KEYS are used as the KEYS of the trajectory message, so the message ID of the target message can be used as the key of the trajectory message to find out related messages from the trajectory topic, and check The message body is parsed. If the message ID field of the parsed message body data matches the target message ID, it is the message trajectory data we want. Generally speaking, under normal circumstances, there should be three sending and consumption trajectories, one sending trajectory and two consumption trajectories (pre-consumption and post-consumption).