Talk about a few of the more common open-source message queuing middleware. If you are doing the message queue technology selection, I do not know choose which message queue, you must first listen to the lesson content.
As a programmer, I believe you must have heard "no silver bullet" this argument, there's a silver bullet means that can easily kill a werewolf with a silver bullet to do, what does that mean? My understanding of this sentence is to say, in software engineering, there is no design, architecture or software as a "silver bullet" which can solve all the problems, every software system, which is unique, you can not use a set of methods to solve all problems.
On the technology selection message queue of the problem is the same reason. He said that does not exist, which message queue is the "best." These common message queue, each product has its own advantages and disadvantages, you need according to the situation existing systems, that you choose the most suitable product.
The basic selection standard message queuing products
Although these message queuing products in terms of features and functionality advantages and disadvantages, but when we choose to have a minimum standard to ensure that the selected product is at least qualified.
Next, let's talk about this pass is what kind of criteria.
First, you must be open-source product, this is very important. Open source means that if one day you use the message queue system encountered an impact on your business Bug, you at least have a chance to quickly fix or circumvent the Bug, to solve the problem you feel the burn of the system by modifying the source code, not helpless to wait for the next version of the developer may not release what time to resolve.
Second, the product must be more popular in recent years, and there is a certain degree of active community products. Popular benefit is that, as long as you are not too popular usage scenario, you encounter Bug probability is very low, because most of you may encounter Bug, others have long met and fixed. Some of the problems you encounter in the course, but also relatively easy to search the Internet to similar problems, and then quickly find a solution.
Another advantage is that popular products with the surrounding ecosystem will have a better integrated and compatible with, for example, Kafka and Flink have good compatibility, Flink built Kafka's Data Source, as it is easy to use Kafka Flink data source development stream computing applications, if you use a relatively small minority of message queuing product, when performing flow calculation, you have to develop a data source Flink of their own.
Finally, as the product of a failed message queue, it must have several characteristics, including:
- Reliable message delivery: ensure not lost message;
- Cluster: cluster support, to ensure that will not lead to a node goes down service is unavailable, of course, the message can not be lost;
- Performance: with good enough performance to meet the performance requirements of the vast majority of the scene.
Next we look at it in line with what these above conditions, the choice of an open source message queuing product.
Choose from the message queue products
1. RabbitMQ
First of all, we talk about old child message queue RabbitMQ, commonly known as rabbit MQ. RabbitMQ is to use a relatively small minority of programming languages: Erlang language, it is the earliest reliable communication between the telecommunications industry system design, is one of several message queue support AMQP protocol minority.
RabbitMQ as the name of the rabbit: a lightweight, fast, it's Slogan, is the slogan, and very clearly indicates that the RabbitMQ features: Messaging that just works, "out of the box with a message queue." . In other words, RabbitMQ is a fairly lightweight message queue, very easy to deploy and use.
In addition RabbitMQ is also known as the world's most widely used open source message queue, is not really the world's first usage, we can not count, but at least "one of the most popular news medium", it is no problem.
RabbitMQ a more distinctive feature is support for a very flexible routing configurations, message queues and other different is that it adds a module Exchange between the producer (Producer) and queue (Queue), you can be understood as a switch.
This effect Exchange module and switches are very similar, according to the configuration of the routing rules is distributed to the sent message producer different queues. Routing rules are very flexible, you can even do it yourself routing rules. Based on the Exchange, which made a lot of play law, if you happen to need this feature, RabbitMQ is a good choice.
RabbitMQ client supported programming language is probably the most of all messages in the queue, if your system is unpopular with some kind of language development, then you can probably find RabbitMQ corresponding to the client.
The next few questions RabbitMQ say next.
The first question is, RabbitMQ message accumulation of support is not good, which in its design, the message queue is a pipeline, a large backlog of messages is an abnormal situation, should try to avoid. When a large number of backlog of messages, RabbitMQ will lead to a sharp decline in performance.
The second question is, is the worst performance RabbitMQ queue of these messages we introduce, according to official data given by empirical testing of the integrated we use every day, depending on the hardware configuration, it probably can handle per second tens of thousands to hundreds of thousands of messages. In fact, this performance is enough to support the vast majority of scenarios, but, if your application message queue very high performance requirements, that do not choose RabbitMQ.
The last question is the programming language Erlang RabbitMQ use of this programming language is not only a very small minority of the language, more trouble is, the learning curve is very steep in this language. Most popular programming languages such as Java, C / C ++, Python and JavaScript, though the syntax, there are a lot of different characteristics, but their basic architecture is the same, you are fluent in only one language, it is very easy to learn other language, even a short time can not proficient, but at least able to achieve the "use" level.
Like an English-speaking people to learn French, German very easy, but if you let him go to learn Chinese, and that basically these learning other languages is not a difficulty level. Unfortunately, Erlang is a programming language "Chinese." So if you want to do some extensions based RabbitMQ and what secondary development, I suggest you seriously consider the issue of sustainable maintenance.
2. RocketMQ
RocketMQ Alibaba queue open source product in 2012, the news was later donated to the Apache Software Foundation, 2017 officially graduated as Apache top-level project. Ali also use RocketMQ as internal support their business message queue, has gone through many "double eleven" test, its performance, stability and reliability are trustworthy. As the excellent home-made message queue, in recent years, more and more are using many domestic manufacturers.
In conclusion, I RocketMQ characteristics, find it hard to RocketMQ What I am particularly impressed with the features, it is difficult to find any shortcomings.
RocketMQ like a good character and scholarship students, has a good performance, stability and reliability with a modern message queue should have almost all functions and features, and it continues to grow in.
RocketMQ have very active Chinese community, most of the problems you can find Chinese answer might be a reason why you chose it. In addition, RocketMQ using Java language development, its contributors are mostly Chinese people, the source code is also relatively easy to read, you can easily be extended to RocketMQ or secondary development.
RocketMQ response delay online business to do a lot of optimization, most cases can be done millisecond response, if your scenario is very concerned about the response delay, it should choose to use RocketMQ.
RocketMQ RabbitMQ higher performance than an order of magnitude per second would be able to handle hundreds of thousands of messages.
RocketMQ is a disadvantage, as the domestic message queue, compared to similar products more popular abroad, and in the international community has not been so popular, and the surrounding ecosystem integration and degree of compatibility to be slightly inferior.
3. Kafka
Finally, we chat Kafka. Kafka was first developed by LinkedIn, currently the top-level Apache project. Kafka was originally designed for processing massive log.
In earlier versions, in order to obtain the ultimate performance, in terms of design done a lot of sacrifices, like not guarantee the reliability of the message, the message may be lost, nor does it support clustering, the function is relatively simple, these sacrifices to handle massive logs this particular scenario is acceptable. Kafka even during this period can not be called a qualified message queue.
However, please note that the focus is generally on the back. Kafka subsequent years gradually filled these short board, you search the Internet to compare a lot of news articles queue is still unreliable, said Kafka, in fact, this argument had elapsed when a. Moment Kafka has developed into a very mature message queue products, both in terms of data reliability, stability and functional characteristics are able to meet the needs of the vast majority of the scene.
Kafka compatibility with the surrounding ecosystem is not one of the best, especially in the large flow of data and computing, almost all of the open-source software system will give priority support related Kafka.
Kafka using Scala and Java language development, extensive use of the idea of the batch and asynchronous design, which makes Kafka can achieve superior performance. Kafka's properties, in particular Asynchronous Receiver performance is best, but RocketMQ no difference in magnitude among the three, can be processed per second to about several hundred thousand messages.
I used to use the case to configure the server to Kafka better the overvoltage test, if there are enough clients concurrent asynchronous batch sent and open compressed, Kafka can limit the processing power of more than 20 million messages per second.
But Kafka problem with this batch of asynchronous design brings a response delay its synchronous messaging is relatively high, because when a client sends a message when, Kafka and will not be sent out immediately, but have to wait a while to save batch and then sent, in its Broker, many places will use this "save the first wave of re-treatment with" design. When your business scenario, the number of messages per second is not so much time, Kafka delay but will be higher. So, Kafka is not suitable for online business scenario.
The second tier message queue
In addition to the above three message queues to introduce you, there are several second-tier products, my personal view is that the reason why these products are not so popular, have more or less obvious short board, not Recommended Use. Here it is, I will brief you, when pure enrich your breadth of knowledge.
Let me talk about ActiveMQ, ActiveMQ is the oldest open source message queue, a decade ago the only alternative open source message queue, has entered old age, the community is not active. Both the functional and performance, ActiveMQ there is a significant gap between modern message queue, meaning it exists only compatible with those grandparents of children still use the system.
Then talk about ZeroMQ, strictly speaking ZeroMQ and can not be called a message queue, but a network-based multi-threaded database message queue, if you need a message queue functions integrated into your system processes, can consider using ZeroMQ.
Finally, talk about Pulsar, many people may have never heard of this product, Pulsar is an emerging open-source message queuing product was first developed by Yahoo, is currently in the growth stage, maturity and relative popularity is not so high. The biggest difference is the other message queue, Pulsar use of storage and computing separation of design, I personally like this design, it has the potential to lead the future development direction of a message queue, it is recommended that you continue to focus on this project.
to sum up
In the understanding of these open source message queue above their own characteristics and advantages and disadvantages, I believe you have to choose the message queue can be aware of. I also summed up a few choice suggestions for your reference.
If we say that the message queue is not what you're going to build one of the protagonists of the system, you do not have high demands on the functionality and performance message queue, only one out of the box for easy maintenance of the product, I suggest you use RabbitMQ.
If your system uses message queue processing is the main online business scene, for example, in using the trading system message queue transfer orders, low latency and level of financial stability that RocketMQ that you need.
If you need to handle vast amounts of information, such as log collection, monitoring information or buried in front of the point of this kind of data, or your application scenario makes extensive use of big data, stream computing-related open source products, that Kafka is best for you message queue.
If I say these scenes and do not meet your scene, you read these messages before I introduce the characteristics of the queue, or do not know how to choose, then choose your most familiar with it, after all these products can meet large most scenarios, using the familiar products also can not quickly get started?