First, what is the message queue
We can compare the message queue is a storage container message when we need to use the message the message can be removed for their own use. Message queues are important components of a distributed system, the use of message queues primarily to improve system performance by asynchronous processing and peak clipping, reduce the coupling system. Currently using more message queues have ActiveMQ, RabbitMQ, Kafka, RocketMQ, we will be 11 later compare these message queues.
In addition, we know that is a FIFO queue Queue data structures, so when consuming messages also follow the order to consume. Producers such as 1,2,3 ... send a message to consumers will be in the order 1,2,3 ... consumption. But the message is occasionally the case of the order of consumption will not occur, such as a news consumer fails or if a message queue can also lead to more consumer spending is not the order, we must ensure that the message is the correct order of consumption.
In addition to the above said problems of consumption order message, use the message queue, we have to consider how to ensure that messages are not repeated consumption? How to ensure the reliability of the transmission of a message (how to deal with the problem of lost messages)? ...... and so on. So the use of message queues is not perfect, it also makes use of system availability, reduce complexity increase, in addition we need to guarantee consistency and other issues.
Second, why should the message queue
I think there are two main benefits of using message queues: 1 by asynchronous processing to improve system performance (peak clipping, reduce the response time required); 2 to reduce coupling system. If you are in the interview the interviewer asked this question, then, in general, is related to the content of your message queue in this area on your resume, this time to recommend you combine your own project to answer.
(1) by asynchronous processing to improve system performance (peak clipping, reduce the response time required)
As shown above, when not in use of the server message queue, requesting the user data directly into the database, in case of a high pressure surge concurrent database, so that response speed becomes slower. However, after using message queues, the user data returned after a request message sent to the queue immediately before the data acquired from the message queue by a consumer process the message queue, asynchronously written to the database. Since message queue database server processing speed (message queue has better scalability than the database), so that the response speed is greatly improved.
Through the above analysis we can conclude that the message queue has a good effect of clipping function - that is, through asynchronous processing, will generate short-term highly concurrent transactional messages are stored in the message queue, thus flattening the peak of concurrent transactions. For example: some of the spike in e-commerce, promotional activities, rational use of message queues can effectively resist the promotion started the influx of a large number of orders for the impact on the system.
Because the user requested data immediately after writing the message queue back to the user, but the request data in a subsequent service may fail verify, write and other operations in the database . So after using message queues for asynchronous processing, require appropriate changes to business processes with , for example, the user after submitting an order, the order data is written to the message queue, the order can not be returned to the user immediately submitted successfully, you need to really deal with the order in the message queue of the consumer process after completion of the order, even after the library, and then notify the user via email or SMS orders successfully , in order to avoid trade disputes. This is similar to mobile phone we usually train tickets and movie tickets.
(2) reduce the coupling system
We know that there is no direct calls between the module if, then add or modify the module on the module to other modules smaller impact, so that the system scalability is undoubtedly better.
Our most common event-driven architecture similar to the producer-consumer model, large sites typically use the message queue with event-driven architecture. As shown below:
The use of message queues publish - subscribe model work, the message sender (producer) announced that one or more message recipients (consumers) a subscription message. From the figure can be seen that there is no direct coupling between the message sender (producer) and the message receiver (consumer) , the message sender to send a message to a message queue that is distributed ends the processing of the message, the recipient message from the distribution after subsequent processing message queue and obtain the message, it does not need to know the message comes from. For new business, as long as interested in that type of news, you can subscribe to this message, no impact on existing systems and services, enabling the scalable business website design .
Recipient of the message the message filtered, treated, packaged, configured as a new type of message, continues to send out a message, wait for other message recipients subscribe to the message. Therefore, based on the event (message object) driven business architecture is a series of processes.
In addition, to avoid the message queue server downtime caused by the loss of a message, the message will be successfully sent to the queue of messages stored on the message server producer, and other consumer after the message is processed real server to delete the message. After the message queue server downtime, server manufacturers will choose other servers distributed message queue server cluster post messages.
Note: Do not think that the message queue can only use the publish - subscribe model work, but in this particular business environment is decoupled using a publish - subscribe model. In addition to publishing - subscribe model, as well as ad-hoc subscription model (only one message a consumer), we more commonly used is released - a subscription model. Further, two JMS message model is provided, the AMQP protocol also provides five message model.
Third, the use of some of the problems caused by the message queue
- Reduce system availability: System Availability reduced to some extent, why do you say? Prior to joining MQ, you do not consider the case of MQ message is lost or hang up and so on, but after the introduction of MQ you need to think about it!
- System complexity increase: After adding MQ, you need to ensure that the message is not repeated consumption, handling of message loss, assure an orderly messaging and so on!
- Coherence: I talked about above can be achieved asynchronous message queues, asynchronous message queues can indeed bring faster system responsiveness. However, if the message is not really correct consumer news consumer how to do? This will result in a data inconsistency!
四、JMS VS AMQP
4.1 SIMILAR
4.1.1 JMS Introduction
JMS (JAVA Message Service, java messaging service) is a java messaging service, the message may be transmitted asynchronously via JMS JMS service between the client. JMS (JAVA Message Service, Java Message Service) API is a standard message service or a specification that allows application components based on JavaEE platform to create, send, receive and read messages. It enables a lower degree of coupling distributed communication, as well as more reliable service message asynchrony.
ActiveMQ is a JMS-based specification implementations.
4.1.2 JMS both messaging models
① point to point (P2P) model
Use queue (Queue) as a message communication bearer; meet the producer and consumer model , a message can only be used by a consumer, the consumer is not the message is retained until the consumer or the timeout in the queue. For example: We are producers send 100 messages, then two general consumers to spend two consumers in order to send a message of their own half of the consumer (that is you an I a consumer.)
② publish / subscribe (Pub / Sub) model
Publish subscribe model (Pub / Sub) using topic (Topic) as a communication bearer message, similar to the broadcast mode ; publisher publishes a message, the message is transmitted to all subscribers through the theme, only after a subscription of the user of the broadcast message the article is not receive the message .
4.1.3 JMS message body five different formats
JMS defines five message body formats, and the message type of call, allows you to send and receive data in a number of different forms, to provide some level of compatibility with existing message formats.
- StreamMessage - the original value of flow data Java
- MapMessage-- a name - value pairs
- TextMessage-- a string object
- ObjectMessage-- a serialized Java object
- A byte stream BytesMessage--
4.2 AMQP
AMQP, namely Advanced Message Queuing Protocol, to provide a unified messaging service application layer standard Advanced Message Queuing Protocol (binary application layer protocol) is an open standard application layer protocol for message-oriented middleware design, compatible with JMS. Based on this agreement with the client messaging middleware can deliver the message, is not limited to the same product, different development languages and other conditions of the client / middleware.
RabbitMQ is based on AMQP protocol.
4.3 JMS vs AMQP
| Contrast direction | ETC. | AMQP |
|---|---|---|
| definition | Java API | protocol |
| Cross-language | no | Yes |
| Cross-platform | no | Yes |
| Support message type | Message provides two models: ①Peer-2-Peer; ②Pub / sub | It offers five models Message: ①direct exchange; ②fanout exchange; ③topic change; ④headers exchange; ⑤system exchange. Essentially, the latter four and JMS pub / sub model is not much difference, only to do a more detailed breakdown on the routing mechanism; |
| Support message type | Support for multiple message types, we mentioned above | byte [] (binary) |
to sum up:
- AMQP defines a protocol message for the wiring layer (wire-level protocol), and JMS API specification is defined. In the Java system, it can be more client interaction, no application code changes by JMS, but its poor support for cross-platform. The AMQP natural cross-platform, cross-language features.
- JMS support TextMessage, MapMessage other complex message type; AMQP supports only the byte [] message type (complex types may be sent after serialization).
- Since routing algorithm Exchange offers, AMQP routing can provide a variety of ways to deliver the message to the message queue, and only supports JMS queues and topics / subscribe ways.
Fifth, the common message queue contrast
| Contrast direction | Overview |
|---|---|
| Throughput | 10,000 of ActiveMQ and RabbitMQ throughput (ActiveMQ worst performance) than one hundred thousand or even one million of RocketMQ and Kafka an order of magnitude. |
| Availability | You can achieve high availability. ActiveMQ and RabbitMQ are based architecture for high availability from the Lord. RocketMQ based on distributed architecture. kafka also distributed, a multiple copies of data, a small number of machine downtime, no data is lost, does not result in unavailable |
| Timeliness | RabbitMQ erlang based development, so the concurrent competence, performance is extremely good, low delay, microsecond. The other three are ms level. |
| Support functions | In addition to Kafka, three other features are relatively complete. Kafka function is relatively simple, the main support simple MQ function, real-time computing and large data log collection in the field is large-scale use, is the de facto standard |
| Message loss | ActiveMQ and RabbitMQ possibility of loss is very low, RocketMQ and Kafka theoretically not lost. |
Reproduced in: https: //juejin.im/post/5d0b97565188255fc6384ecf