A few days ago, Michelle Yeoh asked a question on Knowledge Planet, saying that the company would switch MQ and upgrade from an old service provider to a new service provider, and asked if there is any good plan.
This demand is estimated to be quite common. Here are some experiences to share.
1. Brief description of MQ architecture
As shown in the figure above, the use of MQ asynchronous communication is generally divided into three layers:
Message sender: Use the MQ client to generate messages.
MQ-client::SendMsg(topic, msg);
MQ service: relay messages.
Message recipient: Use the MQ client to consume messages.
MQ-client::RecvMsg(topic, msg, CALLBACK_FUNC);
This is a typical pub-sub architecture. If you want to replace an MQ provider, you need to replace at least three places:
- Sender mq-client
- MQ-server
How to smoothly migrate the receiver mq-client is a topic to be discussed today.
2. Smooth migration plan
The goal of smooth migration is: continuous service and smooth upgrade.
If there are many topics, you need to migrate one by one. The migration of each theme is divided into three steps.
Step 1: Consumer two-way subscription
As shown in the figure above, let us set:
- Pink is the old MQ system
- Blue is the
final goal of the smooth migration of the new MQ system , which is to upgrade the three layers of "publish-service-subscribe" from pink to blue.
The first step is to upgrade the consumer. For the same topic, both the old MQ and the new MQ must be subscribed.
At this time, although there is a TCP connection between "New Service-New Subscription", but "New Release" is not online, no messages will actually be sent (the dotted line in the above figure), and the messages still go through the old MQ (the above figure shows line).
Step 2: The manufacturer upgrades to the new release
The second step is to upgrade the producer, from the old MQ release to the new MQ release.
At this time, a TCP connection will be established between "new release-new service-new subscription", and the message will be transferred to the new channel (solid line in the figure above). Although there is a TCP connection between "old service-old subscription", it is actually not A message will be sent (the dotted line in the figure above).
Step 3: Consumers offline old subscriptions
The third step is to upgrade the consumer, take the old subscription offline, and complete the migration of the entire MQ.
Three, architectural inspiration
MQ changes service providers, ants move, and move smoothly step by step, the cost is actually quite high.
The reason why it is so troublesome and cannot be upgraded in a unified manner is the coupling between the business and the details of the underlying infrastructure (that is, which MQ to use). If the company can "shallow a layer of encapsulation" in the early technical system planning, it can isolate "business code" from "underlying infrastructure details."
Give a more popular example.
If there is no encapsulation layer, the business code is:
ActiveMQ-client::SendMsg(topic, msg);
ActiveMQ-client::RecvMsg(topic, msg, CALLBACK_FUNC);That is, the business side needs to care about ActiveMQ. If the infrastructure is upgraded to RabbitMQ, the business code needs to be upgraded.
If there is a shallow package:
ShenJianMQ::SendMsg(topic, msg){
ActiveMQ-client::SendMsg(topic,msg);
}
ShenJianMQ::RecvMsg(topic, msg,CALLBACK_FUNC)
ActiveMQ-client::RecvMsg(topic,msg, CALLBACK_FUNC);
}The business side does not need to care about the underlying MQ, but only needs to rely on the basic component ShenJianMQ.
At this time, if the infrastructure is upgraded to RabbitMQ, only the basic component ShenJianMQ needs to be upgraded.
The first step: RecvMsg is upgraded to two-way subscription.
ShenJianMQ::RecvMsg(topic, msg,CALLBACK_FUNC)
ActiveMQ-client::RecvMsg(topic, msg, CALLBACK_FUNC);
RabbitMQ-client::RecvMsg(topic, msg, CALLBACK_FUNC);
}Step 2: Upgrade SendMsg to a new release.
ShenJianMQ::SendMsg(topic, msg){
RabbitMQ-client::SendMsg(topic, msg);
}Step 3: RecvMsg offline old subscription.
ShenJianMQ::RecvMsg(topic, msg,CALLBACK_FUNC)
RabbitMQ-client::RecvMsg(topic, msg, CALLBACK_FUNC);
}
You will find that in addition to upgrading and relying on the new version of ShenJianMQ basic components, business code does not need to modify the code.
Not only MQ, the client of the cache and the database, but a shallow encapsulation can also realize the decoupling of the business code and the basic components. When the basic components are replaced or the basic components are upgraded, the business code does not need to be upgraded.
Voiceover: After a shallow package, monitoring/alarming/data collection and other tasks are easier to achieve unified.
Regarding whether to use open source, shallow packaging, or self-research, "Basic components, do you need self-research? "There is a more detailed discussion.
Regarding the smooth migration of MQ, let’s talk so much first, hoping to answer Michelle Yeoh’s question. 
Knowledge Planet-Just started
to play Knowledge Planet, welcome to ask questions.
Voiceover: All questions will be answered carefully (such as this one).
Last time, 200 places were opened and it was full, and then 500 places were opened (not too many people, I am afraid that I can't answer them).
related suggestion:
"Basic components, do I need to research on my own? "
When can't you use MQ"
"When should you use MQ"
"MQ message reachability + idempotence + delay architecture design"
Research:
Does your company have a shallow package? Or self-study? Or is it native coupling?