RabbitMQ is an AMQP implementation, a traditional messaging queue system implementation, based on Erlang. Old-brand MQ products. The AMQP protocol is more used in enterprise systems. For scenarios that require high data consistency, stability, and reliability, performance and throughput are second.
Kafka is an open source MQ system of linkedin. Its main feature is to process message consumption based on the Pull mode and pursue high throughput. The initial purpose is to use for log collection and transmission. 0.8 starts to support replication, does not support transactions, and is suitable for generating large amounts of data. Data collection business for Internet services .
ZeroMQ is just a Pattern library for network programming, which modalizes and componentizes common network request forms (group management, link management, publish and subscribe, etc.), in short, above socket and below MQ. For MQ, network transmission is only a part of it, and more things need to be processed are message storage, routing, Broker service discovery and search, transactions, consumption patterns (ack, re-investment, etc.), cluster services, etc.
RabbitMQ/Kafka/ZeroMQ can provide message queue services, but there are big differences.
In a service-oriented architecture, it is a better idea to use the producer-consumer model for asynchronous communication between services through message brokers (such as RabbitMQ / Kafka, etc.).
Because the dependency between services has changed from strong coupling to loose coupling. The message broker will provide a persistence mechanism, and the message will be saved and will not be lost when the consumer's load is high or the connection is dropped. That is to say, the producer and the consumer do not need to be online at the same time, which is difficult to achieve in the traditional request-response model, and a middleware is needed to do this. Secondly, the message broker can make simple routing strategies based on the message itself, and consumers can perform load balancing and business separation based on this.
There are also disadvantages, that is, the need to build an additional message broker cluster (but the advantages outweigh the disadvantages).
ZeroMQ is different from RabbitMQ/Kafka in that it is just an asynchronous message library that provides a mechanism similar to a message broker based on sockets . If you use ZeroMQ, you need to modify your own business code, which is not conducive to service decoupling.
RabbitMQ supports AMQP (binary), STOMP (text), MQTT (binary), HTTP (packaged in other protocols) and other protocols. Kafka uses its own protocol.
Both Kafka's own services and consumers need to rely on Zookeeper.
RabbitMQ's performance will drop when a large number of messages are accumulated, but Kafka will not. After all, the original intention of AMQP was not designed to persist massive messages, but Kafka was originally used to process massive logs.
In general, RabbitMQ and Kafka are both excellent distributed message broker services. As long as they are deployed reasonably and not done, they can basically meet any demand under production conditions.
ZeroMQ, coordination
RabbitMQ between local processes , working message queue
Kafka, log subscription, focusing on data stream processing
Which of them is more suitable for dealing with distributed transactions? Can it be understood that the veteran RabbitMQ has better support for consistency!
It seems that many companies use zmq to make sockets conveniently. After reading the ZMQ Guide document , my understanding is that this is a series of interfaces similar to Socket. The difference between it and Socket is: ordinary sockets are end-to-end (1:1 relationship), while ZMQ can be N:M relationship. People know more about BSD sockets for point-to-point connections. Point-to-point connections need to explicitly establish connections, destroy connections, and select protocols ( TCP/UDP) and handling errors, etc., while ZMQ shields these details, making your network programming easier. ZMQ is used for communication between node and node, and node can be a host or a process.
But it is indeed a waste of many features of zmq.
RabbitMQ and Kafka are basically the same kind of things, each with its own advantages and disadvantages. ZeroMQ is just a network library and does not support persistence.