First, why the use of message queues?
In most cases, a message queue purpose :: decoupled, asynchronous, clipping .
Through a MQ, Pub / Sub publish-subscribe messaging is such a model, completely decoupled
By the MQ, so that the operation of the synchronization becomes asynchronous operation, improve efficiency.
The number of requests that can be received by MQ, control service
Second, what advantages and disadvantages message queue
There is the advantage that the corresponding benefits, under special scenario decoupled, asynchronous, clipping
There are several disadvantages:
-
Reduce system availability
External system dependencies introduced, the more easy to hang. The need to ensure a highly available messaging queue
-
System complexity increase, to ensure that the message is not repeated consumption, how to deal with the case of message loss, how to ensure that the order of message delivery? The bulk of the big head, a lot of problems, miserably.
-
Consistency
A processing system over direct return is successful, people think that you request will be successful; but the problem is that if the three systems where BCD, BD two write library system is successful, the results of C write library system failed, zezheng? You data is inconsistent.
Third, how to solve the message is lost?
-
ack (consumer confirmation) ( Consumer Manual ACK )
-
Persistence (switches, queues, messages)
-
Does not recommend the use of message transactions will reduce performance verification
-
Manufacturer certification (publisher confirm): After the producer sends a message waiting ACK mq if no failure information is received or received, retry. If you receive a message is the successful conclusion of business.
channel.waitForConfirmsOrDie(10000); -
Reliable Messaging Service (Alternatively, write logic themselves): For some producers do not support acknowledgment message queue, the message may be sent before the message is persisted to the database, and record status message, the subsequent message transmission, are dependent on the consumption processes Analyzing the database and modify the message status.
Fourth, how to avoid the accumulation of messages
-
By the same queue monitor more consumers realize the competition for the message, the message accelerate the rate of consumption.
Fifth, how to ensure the orderly messages?
A: Most service ordered less demanding of the message, if you encounter on the timing requirements of high professional, handled two cases:
-
Concurrent operations while less demanding:
-
Ordered synchronization message is sent when the guarantee
-
To ensure the same queue message is received
-
Ensure a queue only one consumer, there may be a slave (standby state), high availability.
-
Master-slave (Zookeeper cluster primary election)
-
-
Business while concurrently demanding:
-
Said first condition is satisfied scene
-
You can have multiple queues
-
Timing Requirements a set of messages, by dispatching queue to a fixed way hash 2 hash -> klsdilelksd
-
Sixth, how to avoid message duplication of spending?
-
Interface to ensure idempotent, then how to ensure that the interface idempotent it?
-
Some interfaces such as natural power, such as queries
-
Some interfaces are not inherently idempotent, such as new, as well as modify the function of certain interface
1-- received message is a message of 3- 2- been processed
update set stat =3 where id =2 and stat =2
-
It can be determined according to the specific service or states, whether performed by determining the business end of the consumer
msgid transfer message id = 123 - "123456 redis msgid -> 123456, userid
-
-
Seven, Kafka, ActiveMQ, RabbitMQ, RocketMQ what advantages and disadvantages?
| characteristic | ActiveMQ | RabbitMQ | RocketMQ | Kafka |
|---|---|---|---|---|
| Stand-alone Throughput | Ten thousand, lower than RocketMQ, Kafka an order of magnitude | With ActiveMQ | 100,000, supporting high throughput | 100,000, high throughput, large data classes with the general system for real-time calculation data, the log collection and other scenes |
| The impact on throughput topic number | topic can reach levels of hundreds / thousands of throughput will decrease to a lesser extent, this is RocketMQ a big advantage, in the same machine, you can support a large number of topic | topic from tens to hundreds of times, throughput could drop significantly, in the same machine, Kafka try to ensure that the number of topic not too much, if you want to support a large-scale topic, need to add more machine resources | ||
| Timeliness | ms level | Microsecond, which is a major feature of RabbitMQ, lowest latency | ms level | Within ms delay stage |
| Availability | High, based on a master-slave architecture to achieve high availability | With ActiveMQ | Very high, distributed architecture | Very high, multiple copies of distributed data, a small number of machine downtime, no data is lost, it does not result in unavailable |
| Message reliability | Have a lower probability of data loss | Basic is not lost | After parameter optimization configuration, it can do 0 loss | With RocketMQ |
| Support functions | MQ extremely functional areas of complete | Based on the development of erlang, concurrency is very strong, excellent performance, low latency | MQ function more perfect, or distributed, scalability | Function is relatively simple, the main support simple MQ function, real-time computing, and log data collected in the field is large scale use |
Small and medium companies , technical strength is more general, the technical challenges are not particularly high, with RabbitMQ is a good choice;
Large companies , R & D infrastructure stronger, with RocketMQ is a good choice.
If it is a large area of data