RocketMQ copy code eating record

1. Background

The most recent business scenario requires data synchronization. Simply put, the source data must be stored in the Oracle database. There are too many historical data in the Oracle database, and the query takes too long to be accepted by users. Therefore, the mysql database is divided into tables and the data must be synchronized to the mysql database. The original implementation was to run a scheduled task every day to synchronize the data of the previous day. The problem came, and the data was not synchronized in time. If you query, there are two interfaces. The historical data is checked by Mysql, and the data on the day is checked by Oracle. But the logic of the two interfaces is exactly the same, because the business functions are the same, which is very **[difficult to describe]. So the point is here. Using the message method, when the data is saved to Oracle, send a message, the message body is the saved data, consume the message, and save the data to Mysql, which basically guarantees real-time performance. What if you don’t use messages? After saving Oracle, and then saving to Mysql, is this saving interface too long? Brother dei [not

Second, realize

1. Message producer configuration

/**
 * 消息生产方配置
 *
 * @yx8102 2019/1/25
 */
@Component
public class MessageProducerConfig {

    // 消息服务器地址
    @Value("${message.producer.serverAddress}")
    private String serverAddress;

    // 发送分组名称
    @Value("${message.producer.groupName}")
    private String groupName;

    @Bean
    public RocketMqClient initRocketMqClient(){
        validateMessage();
        RocketMqServer server = RocketMqServer.build(this.serverAddress);
        return RocketMqClient.build(server, this.groupName);
    }

    private void validateMessage(){
        Assert.notNull(this.serverAddress, "serverAddress is blank");
        Assert.notNull(this.groupName, "groupName is blank");
    }

    public void setServerAddress(String serverAddress) {
        this.serverAddress = serverAddress;
    }

    public void setGroupName(String groupName) {
        this.groupName = groupName;
    }
}

2. Send message service

/**
 * 发送消息服务
 *
 * @xyang010 2019/1/30
 */
@Service
@Slf4j
public class PictureSyncMessageService {

    @Autowired
    private RocketMqClient rocketMqClient;


    /**
     * 发送同步消息
     * @param syncFailedDO
     */
    public void sendPictureSyncMessage(PictureSyncFailedDO syncFailedDO) {

        //1. 拼装消息
        RocketMqMessage mqMessage = new RocketMqMessage();
        mqMessage.setTopic(MessageConstant.ROCKET_MQ_TOPIC);
        mqMessage.setTag(MessageConstant.PICTURE_SYNC_TAG);
        mqMessage.setKey(buildMessageKey(syncFailedDO.getZpFileId()));
        mqMessage.setData(syncFailedDO);

        //2. 发送消息
        ITryCatchResult<SendResult> invokeResult = rocketMqClient.send(mqMessage);

        //3. 发送成功
        if(invokeResult.isSuccess() && invokeResult.getResult() != null) {
            SendResult sendRes = invokeResult.getResult();
            SendStatus sendStatus = sendRes.getSendStatus();
            if (sendStatus.equals(SendStatus.SEND_OK)) {
                
            }
        }
    }

    /**
     * 创建messageKey
     * @param id
     * @return
     */
    private static String buildMessageKey(Long id){
        return  "zpFileId:"+ id +"_" + UUIDUtils.getUUID("N");
    }
}

3. Consumer message configuration

/**
 * 消息消费方配置
 *
 * @yx8102 2019/2/2
 */
@Component
@Slf4j
public class MessageConsumerConfig {

    @Value("${message.consumer.serverAddress}")
    private String serverAddress;

    @Value("${message.consumer.groupName}")
    private String groupName;

    @Value("${message.consumer.topic}")
    private String topic;

    @Autowired
    private PictureSyncMessageListener messageListener;

    @Bean
    public DefaultMQPushConsumer getRocketMQConsumer() {
        validateMessage();
        DefaultMQPushConsumer consumer = new DefaultMQPushConsumer(groupName);
        consumer.setNamesrvAddr(serverAddress);
        consumer.setMessageListener(messageListener);
        consumer.setConsumeMessageBatchMaxSize(10);
        consumer.setConsumeThreadMax(10);
        consumer.setConsumeThreadMin(5);
        try{
            consumer.subscribe(topic, MessageConstant.PICTURE_SYNC_TAG);
            consumer.start();
            log.info("message consumer is start successfully");
        } catch (MQClientException e){
            log.error("message consumer start error {}", e);
        }
        return consumer;
    }

    public void setServerAddress(String serverAddress) {
        this.serverAddress = serverAddress;
    }

    public void setGroupName(String groupName) {
        this.groupName = groupName;
    }

    public void setTopic(String topic) {
        this.topic = topic;
    }

    private void validateMessage(){
        Assert.notNull(this.serverAddress, "serverAddress is blank");
        Assert.notNull(this.groupName, "groupName is blank");
        Assert.notNull(this.topic, "topic is blank");
    }
}

4. Consumer messaging service

/**
 * 消息监听服务
 *
 * @yx8102 2019/2/2
 */
@Component
@Slf4j
public class PictureSyncMessageListener implements MessageListenerConcurrently {

    @Override
    public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> msgs, ConsumeConcurrentlyContext context) {

        for (MessageExt messageExt : msgs) {
            String message = new String(messageExt.getBody());
            log.info("receive picture sync message {}", message);

            PictureSyncFailedDO syncFailedDO = JSON.parseObject(message, PictureSyncFailedDO.class);
            
            return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
        }
    }
}

5. Description

• Increase consumer configuration: clusterName, brokerName, groupName
• Add topic: use camel case naming
• Just use one producerGroupName and consumerGroupName for each project of the producer and consumer
• The same topic can be filtered according to different tags
• The producer can only start once
• Consumers must remember to start
• In the local single test, the sleep time is long enough to ensure that the message is sent and monitored
• ConsumerListener can be configured to the corresponding class, and different topics need different configuration classes