Message Queue - The queue you need to use it?

      For message queues, should not unfamiliar. On the market many types of message queues, varied. Choose their own message queue for the scene is not easy. How to choose? This article will not be a simple comparison of the pros and cons of all the queues, the way will be from its own advantages and disadvantages message queue, so that we do not need to feel the need to use the message queue.

 

Message queue architecture model

      Message definitions Agent: Agent message is an architectural model for message authentication, conversion, routing. Adjusting a communication application, minimizing perceptual each other (dependent), effective to achieve decoupling. For example, a message broker can manage workload queue or queue, a plurality of receivers, to provide reliable storage, to ensure that the message distribution, and transaction management. Wikipedia ( https://zh.wikipedia.org/wiki/%E6%B6%88%E6%81%AF%E4%BB%A3%E7%90%86 ).

      Simple, message broker architectural pattern is a message queue, the message queue will generally be used. Including: ActiveMQ, Kafka, RocketMQ, RabbitMQ and so on.

 

Message Agent way

      Message broker ways: direct point to point, publish-subscribe, custom routing rules.

 

Direct point to point                            

      In this way, the producers send (put) a message, a message direct consumer consumption. In the message broker perspective, positioning routing by the queue name, producers and consumers only need to know the queue name. As shown below, it shows a producer into a message queue, a responsible consumer spending.

      Of course, if the traffic flow, multiple producers and consumers. Since message queue uses the same name, the multiple instances can be added in the production and consumption of the application side, improve processing speed. The following figure, the producer A, B are responsible for same queue, and consumers A, B responsible for fighting for the same queue, who scramble to whoever consumption current message.

      There are scenarios in this way include: e-mail, text messaging, video processing, image processing, and third-party applications such as docking.

 

Publish and subscribe                                   

      Publish-Subscribe way is very common, most frequently used. In this way there are producers and consumers, the producers put a message into the message queue of a topic, if you want to get this message then you can subscribe to this message. That consumer subscriber, you can subscribe to multiple consumers have the same theme. In this way together with one difference is that, when ABC producers put a message to the topic X, if you subscribe to a topic X by two consumers, then the consumers will receive the message ABC. If there are n consumers subscribed to the topic X, then there will be n number of consumer spending this message. As shown below.

      In this way, the queue is how to deal with it? After the news ABC producer put into the subject X, the message queue consumers subscribed to know that there are two themes X. ABC news message queue will then more than one copy, ie a total of two. Then send the message to each of the subscribers in the respective. As shown below.

      Release subscription model has many application scenarios, such as the demand that a message needs batch will be used this way. Such as statistics and news of success under a single push. That mall has orders demand, is finished after a single information required under a single SMS message push and statistics.

      Subscribe to publish the benefits that the consumer end scalability is very strong. Subscribers can continue to increase, as in the example above, if coupled with a motion to send email, then do not affect the original, based on increased consumer email C.

 

Custom routing                                        

      This approach is a queue can be customized routing through customized routing decide how to route messages to the queue.

      With respect to the above two ways, customized routing less frequently used, but there will be a scene to use. Not all queues are supported in this way. Custom routing flexibility can enhance consumers subscribe messaging, enabling the system can be used to configure implement the new requirements.

There are three message protocol AMQP, STOMP, JMS. Not here in detail.

 

Advantages of the message queue

      There are advantages of asynchronous processing, clipping, increase availability, decoupling.

Asynchronous Processing                                             

      异步处理是消息队列使用得最多的方式之一。异步处理，可以是线程，可以是硬件，也可以是队列。这里当然说的是队列。队列异步处理专门用于耗时处理等资源密集型工作，任何缓慢的操作都可以考虑使用消息队列来做异步处理。

      异步处理的应用场景有：发短信、发eamil、图片处理、视频处理、生成excel等。

 

削峰                                                   

      削峰提高了应用程序的可用性。使程序能平稳运行。削峰也叫流量削峰，旨在流量超过程序处理时让流量平稳地在应用程序中处理。在队列的削峰处理中，生产者将所有的流量进入队列，队列的消费者一个一个地平缓进行处理，多余的消息在队列中。为系统的可能性提供了保障，避免了过大流量进入而影响到系统宕机等意外情况发送。

      即使系统无法立即处理所有请求也不会崩溃。峰值过后，系统很快自动回复到平常状态。如下图（来自网络）。

提高可用性                                            

      提高可用性指的是提高应用程序的可用性。在消息队列里有生产者与消费者，他们之间无需知道对方的位置在哪里，只需知道消息的协议、消息类型格式即可。所以，当我们的生产者部署的应用与消费者部署的应用不相同时，可提高可用性。即消费者所在应用宕机或生产者所在应用宕机，都不会影响对方，应用也不会崩毁。当你部署多个生产者或者消费者的应用时，其中一台宕机也不会影响整个应用程序，只会影响吞吐量而已。

解耦                                                     

      这里的解耦是指隔离了队列的生产者与消费者。

      The following scenarios, you need to send a verification code to verify the authenticity of the email mailbox user registration is successful. Register here to provide an API interface layer, this first API generates user information, and then call queue queue to tell the producers I need to send email. Finally, the consumer receives the end of a program message and an email. Here consumers and producers isolation, decoupling the producers and consumers, who generate just need to put the message queue, consumers only need to wait for notification, if the consumer notice and send email.

 

Challenges and shortcomings of the message queue

      Message Queue easy to use, but it also brings challenges. One is an asynchronous message queues for the program more difficult and time.

      Sync one-stop processing before it, and now must be separated asynchronous producers and consumers. Thus increasing the difficulty and development time.

      The second is to ensure that the order of the message. Message ordering guarantees is a major issue, a lot of internal message queues have been supported. On the application design, to avoid the ordering of the message.

      The third is idempotent. Many requirements need to retry the message queue, so make sure idempotency Some retry queue. Idempotent on the application may be controlled to ensure that the multiple queue processing time consumption.

 

to sum up

      Message to realize extremely complex internal queue on the market regardless of the application or open source programs, have brought us their convenience.

      We do not need to need to use the queue, consider the need for asynchronous processing, clipping, improve the availability of the program, decoupling program. If you only consider the asynchronous processing is also be implemented in other ways, only one queue method.

      Queue bring us many benefits, but also need to consider the challenges brought to us, including the challenges of development time and increase the difficulty to avoid news sequential dependencies idempotency.

      So we consider the queue before use, do all the research to see if it is worth. General benefit of the queue is far greater than the disadvantages.

 

No. I can focus on the public, many years of experience of the original article for everyone to share.