Foreword
Message Queue has gradually become the core of enterprise IT means of internal communication systems. It has a low coupling, reliable delivery, broadcast, traffic control, eventual consistency and a series of functions, one of the main means of asynchronous RPC. There are many of today's mainstream messaging middleware, such as veteran's ActiveMQ, RabbitMQ, hot Kafka, Alibaba developed RocketMQ and so on. Today mainly describes the differences and to contact the next few mainstream messaging middleware.
1. Introduction --ActiveMQ mainstream messaging middleware
ActiveMQ is produced by the Apache, ActiveMQ is a fully JMS1.1 specification and J2EE 1.4 JMS Provider implementation. It's very fast, supports multiple languages and client protocols, and can be very easily embedded into enterprise application environment and has many advanced features.
1.1 Features
- Apache ActiveMQ is produced, most popular, strong ability to open source message bus, and it is a fully supported JMS messaging middleware specification
- Its rich API, multiple cluster mode enabled him to build an industry veteran messaging middleware, widely used in small and medium enterprises!
- MQ metrics: service performance, data storage, cluster architecture.
ActiveMQ is now less used because ActiveMQ compared to other MQ performance is more general. Now high concurrency scenarios, big data can be seen everywhere. If this time on the MQ choice, then ActiveMQ becomes insufficient.
MQ measure a target, there are three main areas: service performance, data storage, cluster architecture service performance: ActiveMQ performance is not particularly good, the face of large scale concurrency, always will be a variety of small problems, such as blocked , excessive accumulation of messages, some problems of some delays and so on. Data storage: ActiveMQ defaults KahaDB memory storage. May be used in some high-performance storage, such as: google LevelDb based on the stored within c. If the message is to ensure reliable, or may be used mysql oracle database. Cluster architecture: ActiveMQ popular for so many years, integration with other components Api is also very perfect. If it is not particularly large under concurrency scenarios, ActiveMQ is also a good choice. Because ActiveMQ cluster architecture model is also very good.
1.2 architecture patterns
NetWork mode
Essentially two integrated standby mode, with intermediate NewWork gateway, to make a connection configuration, you can achieve distributed cluster.
1.3 Summary
advantage:
- Cross-platform (written in JAVA platform-independent, ActiveMQ can run on almost any JVM)
- You can use JDBC: data can be persisted to the database. Although ActiveMQ using JDBC can degrade performance, but the database has been familiar to most developers storage media
- JMS specification support: Support for JMS specification provides a unified interface
- Support automatic reconnection error and retry mechanism
- There are safety mechanisms: supports shiro, jaas and other security configuration mechanism to authenticate and authorize the Queue / Topic
- Monitoring perfect: have a sound monitoring, including WebConsole, JMX, Shell command line, Jolokia's RESTful API
- Friendly interface: WebConsole provided to meet the majority of cases, there are many third-party components can be used, such as hawtio
Disadvantages:
- Community activity not as high RabbitMQ
- According to other user feedback, it will be a baffling problem, the message will be lost
- Currently the focus put activemq6.0 products Apollo, less maintenance for the 5.x
- Not suitable for application scenario thousands of queues
2. mainstream messaging middleware introduction --Kafka
Apache Kafka is a distributed publish-subscribe messaging system. It was originally based on the LinkedIn's unique design is implemented as a distributed system logs submitted (a distributed commit log), later to become part of the Apache project. Kafka performance, efficiency, and may be extended good persistence. Its partitioning feature, you can copy and fault tolerance are its good characteristics.
2.1 Features
kafka LinkedIn is an open source distributed release - scheduled for messaging system, currently belongs to the top-level Apache project. Kafka main characteristics is given Pull model to deal with consumer messages, the pursuit of high throughput, the purpose is to start for log collection and transmission. 0.8 version began to support replication, does not support transactions, repeated message is lost, the error is not strictly required, suitable for generating a large number of Internet data collection business data services. Here you can see kafka focus only on throughput. Therefore, when using kafka, pay attention to whether the service allows messages to be repeated, lost errors. If allowed, kafka is the most appropriate. Because its performance is the highest. Even on inexpensive servers, the amount of data can support more than 100k per second stand-alone. So it's performance is very good. kafka using only memory for storing, as long as there is sufficient memory, can be sufficiently large throughput. Because kafka not read and write on the disk.
- Fast Persistence: the message may be persistent in O (1) of the overhead;
- High Throughput: on a common server throughput rate can be achieved either 10W / s of;
- Fully distributed systems: Broker, Producer and Consumer are native support for distributed automatic, automatic load balancing;
- Support synchronous and asynchronous replication two kinds of high-availability mechanisms;
- Batch support data transmission and pulling;
- Zero-copy technique (zero-copy): IO reduction steps to improve the system throughput;
- Data migration, expansion transparent to the user;
- Without downtime extension machine;
- Other features: a wealth of information pull model, efficient subscribers horizontal scaling, real-time news subscription, one hundred million messages accumulation capacity, periodically delete mechanism
2.2 architecture patterns
kafka architecture model
Zookeeper depends coordinated management, can every kafka Replicas is data synchronization. If you say: There is a data falls on the first node, it will be repilicate copy, so there is a run in each node data, there are a total of three points of data. If one of them goes down, but also from the other two nodes acquired data. Deployment Scenario Suggestion: deploy across the room. Even if there is a machine goes down, the data is no problem. If the entire site is down. So our data will be lost. This is the big companies need to consider off-site disaster recovery . Of course kafka focused performance, the reliability and high data concerned.
2.3 Summary
advantage:
- Rich client language: support for Java, .Net, PHP, Ruby, Python, Go and other languages;
- High performance: single write about 1,000,000 TPS / sec, message size of 10 bytes;
- It provides fully distributed architecture, and a replica mechanism, with high availability and reliability, theoretically support unlimited messages pile up;
- Support batch operations;
- Consumers using Pull way to get the message. Messages ordered by controlling to ensure that all messages are consumed and only be consumed once;
- There are excellent third-party KafkaWeb management interface Kafka-Manager;
- More mature fields in the log, and more companies are using open source projects.
Disadvantages:
- When Kafka more than 64 stand-alone queue / partition will happen obvious soar phenomenon Load. More queues, the higher the load, the response time becomes long message transmission;
- Use short polling mode, depending on the real-time polling interval;
- Consumer failure does not support retry;
- Support message sequence, but after a down proxy, the message is generated out of order;
- Community Update slower.
3. mainstream messaging middleware introduction --RocketMQ
RocketMQ Ali open source messaging middleware, is also already incubator for Apache top-level project. The Java language, designed with reference to Kafka, and made some of their own improvements, the reliability of the news better than Kafka. RocketMQ internal Ali is widely used in order, transaction, recharge, stream computing, push notifications, log streaming, binglog distribution and other scenes.
3.1 Features
The core features are as follows:
- Ensure message sequence, message sequence consumer.
- It provides a wealth of pulling and processing mode.
- Efficient subscribers can scale horizontally.
- Carrying millions accumulation ability level messages.
3.2 architecture patterns
RocketMQ cluster architecture model 1.Master-Slave (from the master) mode 2. Dual Master mode. 3. Dual dual master-slave mode. 4. A multi-master multiple-slave mode. 5. from a multi-master mode. Many options to choose from.
And so on, a lot of open source reference design mode.
Cluster topology
3.3 Summary
advantage:
- Stand-alone support more than 10,000 persistent queue;
- All messages RocketMQ are persistent, first writing system PAGECACHE, then brush disk, you can ensure that the disk has a data memory, and to access, read directly from the memory.
- Model is simple, easy to use interfaces (JMS interface on many occasions and not very practical);
- Performance is very good, you can allow the accumulation of a large number of messages in the Broker;
- It supports a variety of consumption patterns, including the cluster consumption, radio consumption;
- All aspects of distributed scalable design supports master-slave and availability;
- Development of a more active version of the update soon.
Disadvantages:
- Supported client language much, Java and C ++ is now, which is not mature C ++
- Maintenance RocketMQ need a professional team
- Business Edition charge, there are many features that are not provided externally.
- Does not implement MQ JMS interfaces in the core in
4. Why RabbitMQ?
1.ActiveMQ, performance is not very good, so in highly concurrent scenarios directly pass out. It Api perfect, you can go to the Internet to use in small and medium sized companies. 2.kafka, the main emphasis on high performance, when the business needs if the reliability of the delivery of the message. You will not be able to select the kafka. But if you do some log collect it, kafka is very good. Kafka because the performance is very good. 3.RocketMQ, it features very good. It is high performance to meet the reliability, distributed things, the level of support the expansion, the level of hundreds of millions of messages accumulate, switching between master and slave, and so on. MQ all the advantages of it are basically satisfied. But its biggest drawback: the commercial version of the charges. So it has many features that are not provided externally.
Having then there is no other three MQ MQ can choose it? Yes, this is also a learning MQ - RabbitMQ.
The mainstream messaging middleware introduced --RabbitMQ
RabbitMQ released in 2007, was completed in a AMQP (Advanced Message Queuing Protocol) based on reusable enterprise messaging systems, it is one of the most mainstream messaging middleware.
5.1 Features
RabbitMQ is an open source message queuing system with Erlang language development to achieve based on AMQP protocol. AMQP main feature is a message for the queue, the routing (including point and publish / subscribe), reliability and safety. More AMQP protocol used within the enterprise system, data consistency, high stability and reliability requirements scenario, the performance and throughput requirements are still followed. RabbitMQ's reliability is very good, to ensure that data is not lost hundred percent. Mirroring the queue can be used, its stability is very good. So the Internet in our financial sector. Under the stability and reliability of the data requirements are very high, we would choose RabbitMQ. Of course not kafka good performance, but the performance is much better than AvtiveMQ. You can also do some of his optimize performance. RabbitMQ live remote dual architecture can be constructed, each node comprising a disk or memory storage methods can be used.
RabbitMQ cluster architecture
Figures say is that we can use as a set of three nodes clustered RabbitMQ, of course, can have many groups. Using mirror queue between the node and the node. Based on this, it is possible to ensure that data is not lost one hundred percent. Front-end load balancing can do, such as load balancing components: HA-proxy, a TCP-level load. If you want a highly available, then you need the help of a high-availability configuration keepAlived do. For example add a virtual front end VIP, VIP routed through the specified load balancing assembly, then it has a route to a node of RabbtMQ. This is the entire RabbitMQ cluster architecture. Can achieve very complete, high availability and performance is also very good, super stability. And there are various cluster recovery means. For example: one of the nodes hung up, or a disk is damaged, it can also be a message repair. Based on these advantages, we must take RabbitMQ learn.
6. Comparison of FIG.
The end of the sentence
Welcome attention to personal micro-channel public number: Coder program for the latest original technical articles and free learning materials, a lot more boutique mind maps, interview data, PMP preparation materials waiting for you to lead, allowing you to learn technical knowledge anytime, anywhere! Create a qq group: 315 211 365, we welcome into the group exchange study together. Thank you! Can also be introduced to the side of a friend in need.
Articles included to
Github: github.com/CoderMerlin…
Gitee: gitee.com/573059382/c... welcome attention and star ~
Reference article:
"RabbitMQ messaging middleware succinctly"
recommended article:
RabbitMQ (a) Windows / Linux environment to build (full version)