First, what is the message queue?
Message queues do not know if you see this word, will not feel it is a relatively high-end technology, anyway, I think it seems to be quite fast hardware.
Message queues, we typically referred to it as MQ (Message Queue), ah, that is very straightforward shorthand.
We first whether the message (Message) the word, to look at the queue (Queue). This look, queue we should all be familiar with it.
A queue is a FIFO data structure.
In fact, Java inside, a lot has been achieved in the queue:
Why does it need a message queue (MQ) middleware do this? ? ?
In fact, this problem before when I studied with the Redis like. Redis is a database stored in a memory key-value form, obviously we could use something like this HashMap implementation class can achieve a similar effect, and it is also why Redis? "Redis Collection"
Here, you can guess why should first message queue (MQ) this middleware , the following will continue to add.
Message queue can be simply understood as: the data to be transmitted in a queue .
Popular Science:
The data into the message queue called the producer
fetch data from the message queue called the inside consumers
Second, why should the message queue?
Why use a message queue, that is, Q: What message queue with the benefits. We look at the following scenario
2.1 Decoupling
Now I have a system A, system A can generate a userId
Then, there are systems B and C systems will need to do this userId related operations
Pseudo code may be written in this: public class SystemA { // systems B and C systems rely SystemB = systemB and new new SystemB (); SystemC systemC = new new SystemC (); // System A data unique to the userId Private String the userId = "Java3y" ; public void doSomething () { // systems B and C systems are required to hold the operating system a userId other things systemB.SystemBNeed2do (userId); systemC.SystemCNeed2do (userId); } }
ok, everything would spend a few days.
One day, the person in charge told the person in charge of system B system A and system B is now SystemBNeed2do (String userId) This interface is no longer used, let the system do not go to A tune it .
Thus, the person in charge of system A, said, "Okay, I will not call you.", So he took the calling code B interfaces to the system deleted :
public void doSomething () { // System A System B interfaces are no longer called // systemB.SystemBNeed2do (the userId); systemC.SystemCNeed2do (the userId); }
A few days later, the person in charge of the system D received a demand, but also need to use userId system A, then ran to talk to the person in charge of system A, said: "My brother, I have to use your userId, you tune my interface it. "
A system then said: "No problem, which engage in"
Then, the code system A is as follows:
public class SystemA { // no longer necessary to rely on the system B // SystemB systemB and new new SystemB = (); // System C and System D dependent SystemC systemC = new new SystemC (); SystemD systemD = new new SystemD (); // system a unique data Private String the userId = "Java3y" ; public void doSomething () { // no longer necessary to rely on the system B // systemB.SystemBNeed2do (the userId); // system C and system D are a need to hold the system to operate userId other things systemC.SystemCNeed2do (userId); systemD.SystemDNeed2do (userId); } }
time flies:
A few days later, the person in charge of the system E over, tell the system A, needs userId.
A few days later, the person in charge of system B came up and tell the system A, it was again out of that interface.
A few days later, the head of System F's over, tell the system A, needs userId.
......
So who is responsible for system A, to this day have been harassed, to change to change, to change to change .......
There is another problem, call the system C when C if the system hung up, system A but also how to deal with. If you call the system D, due to network delays, the request times out, and that is the feedback system A fail or retry? ?
Finally, the person in charge of system A, that from time to time to change to change, boring, then ran up the road.
Then, the company attracted a big brother, big brother familiar after a few days, came up and said: The userId system A written message queue, so that the system would not often change the A .
why? Let's take a look:
A system userId writes the message queue, the system C and D to take the data from the system message queue. So what are the benefits?
A system is only responsible for the data written to the queue who want or do not want this data (information), system A do not care about.
Even now System D do not want userId this data, the system B userId suddenly want this data, and has nothing related to system A, system A bit of code do not change.
System D userId not take through the system A, but from the message queue to get inside. System D or even hang request times out, regardless of the system related to A, with only the relevant message queue.
Thus, System A and System B, C, D are decoupled from the.
2.2 asynchronous
Let us look at this situation the following: A system or directly call systems B, C, D
code show as below:
public class SystemA { SystemB systemB and = new new SystemB (); SystemC systemC = new new SystemC (); SystemD systemD = new new SystemD (); // System A unique data Private String the userId; public void doOrder () { // orders = userId the this .order (); // If the order is successful, the arrangements for other systems to do something systemB.SystemBNeed2do (userId); systemC.SystemCNeed2do (userId); systemD.SystemDNeed2do (userId); } }
A system userId calculated assuming specific values need to 50ms, the system call interface B needs to 300ms, the system call interface requires C, 300ms, D interface call system needs 300ms. Then this request will need 50 + 300 + 300 + 300 = 950ms
And we know that the system A is doing major business , and the system B, C, D are non-main business. A system such as dealing with single orders , while orders for single-system B is successful, then send a text message to tell the user this specific order has been successful, and System C and System D is also dealing with some trivial matter.
So this time, in order to improve the user experience and throughput , in fact, can asynchronously call the system B, C, D interface. So, we ended up like this:
A system after execution is completed, the written message queue userId, and then return directly (For other operations, the asynchronous process).
Originally the entire request need 950ms (sync)
will now be calling other systems of asynchronous interfaces, only 100ms (asynchronous)
(examples might give it very well, but I think the explanation to the point on the line, forgive me.)
2.3 clipping / limiting
We come back a scene, we now engage in a big promotion a month, during concurrent big promotion may be very high, such as 3000 requests per second. Suppose we now have two machines to process the request, and each machine can only handle 1000 requests each time.
That extra 1000 request, we may put the entire system to collapse out ... So, there is a way, we can write the message queue:
System B and C systems according to their number of requests that can be processed to get the data in the message queue , so even if there are 8,000 requests per second, it is only the request in the message queue, the message queue to get to by the system itself control , so as not to engage the whole system will collapse.
Third, the use of message queues have any questions?
After our scenario above, we can already find the message queue can do actually still find many.
Here, let's back to the beginning of the article, "obviously has a lot of JDK queue achieved, we also need a message queue middleware it?" Is actually very simple, kind of queue JDK implementation, although there are many, but all is simple memory queue . Why do I say JDK is a simple memory queue it? Let us look to achieve Message Queue (middleware) What questions might want to consider.
3.1 High Availability
Whether we use a message queue to do decoupling, asynchronous or clipping, message queues certainly can not be a single of. Try to think about, if it is a stand-alone message queue, if the machine hung up, that our entire system is almost unusable.
So, when we project using message queues are too clustered / distributed. Do cluster / distributed is bound to hope that the message queue can provide a ready-made support, rather than trying to manually write code to achieve.
3.2 data loss
We will write data to the message queue, B and C have not had time to take the message data queue, and hung up. If you do not do any measure, our data is lost .
I learned Redis knows, Redis data can be persisted on disk, in case Redis hung up, but also to recover data from disk from over. Similarly, the data in the message queue also needs to exist somewhere else, so as to reduce the loss of data as possible.
What does that exist?
Disk?
database?
Redis?
Distributed File System?
Synchronous or asynchronous memory storage?
3.3 how consumers get the data message queue?
How consumers get the data from the message queue inside? There are two ways:
Producer data into the message queue, the message queue data, and the initiative to call the consumer to pick up ( commonly known as the Push )
consumers continue to go in rotation message queue, see if there is new data, if you consume ( commonly known as pull )
3.4 Other
In addition to these, we use the time to have to consider a variety of issues:
Repeat the message consumer how to do ah?
I want to make sure the message is absolutely sequential how to do?
...... ..
Although the message queue has brought us so many benefits but we found that the introduction of a message queue will also increase the complexity of the system. Now the market has a lot of wheels the message queue, each message queue has its own characteristics, choose what kind of MQ have good discretion .
Original link: https: //blog.csdn.net/yue_2018/article/details/89305275