Article directory
Table of contents
2.2.2 Work queues work queue mode
2.2.3 Pub/Sub subscription model
foreword
The full name of MQ is Message Queue ( message queue ) , which is a container for storing messages during message transmission. It is mostly used for communication between distributed systems.
MQ, message queue, middleware for storing messages. There
are two ways of distributed system communication: direct remote call and indirect communication with the help of a third party. The
sender is called the producer, and the receiver is called the consumer.
Advantages and Disadvantages of MQ
Advantages
Application decoupling: Improve system fault tolerance and maintainability
Asynchronous speed up: Improve user experience and system throughput Peak shaving and
valley filling: Improve system stability
Disadvantages
Reduced system availability: The more external dependencies introduced by the system, The worse the system stability. Once MQ goes down, it will affect the business. How to ensure the high availability of MQ Increased
system complexity: The addition of MQ has greatly increased the complexity of the system. In the past, there were synchronous remote calls between systems, but now asynchronous calls are made through MQ. How to ensure that messages are not consumed repeatedly? How to deal with message loss situation? Then ensure the order of message delivery. Consistency
problem: System A sends message data to three systems B, C, and D through MQ. If system B and system C process successfully, system D fails to process. How to ensure the consistency of message data processing?
1. Brief introduction
Since MQ has advantages and disadvantages, what conditions need to be met when using MQ?
Producers do not need to get feedback from consumers. Before the introduction of the message queue, the return value of the interface should be empty. This makes it possible for the upper layer to continue to move forward as if the action of the lower layer has not been performed, which is the so-called asynchronous.
Temporary inconsistencies are tolerated.
It is indeed effective. That is, the benefits of decoupling, speed-up, and peak shaving exceed the costs of joining and managing MQ.
At present, there are many MQ products in the industry, such as RabbitMQ, RocketMQ, ActiveMQ, Kafka, ZeroMQ, MetaMq, etc., and there are also cases of directly using Redis as a message queue. Comprehensive consideration of own needs and MQ product features
AMQP , or Advanced Message Queuing Protocol (Advanced Message Queuing Protocol), is a network protocol, an open standard for application layer protocols, designed for message-oriented middleware. The client and message middleware based on this protocol can transmit messages, and it is not limited by different client/middleware products, different development languages and other conditions. In 2006, the AMQP specification was released. Analogy HTTP .
Related concepts in RabbitMQ:
Broker : an application for receiving and distributing messages, RabbitMQ Server is Message Broker
Virtual host : designed for multi-tenancy and security factors, divide the basic components of AMQP into a virtual group, similar to the network namespace concept.
When multiple different users use the services provided by the same RabbitMQ server, multiple vhosts can be divided, and each user creates a connection such as exchange/queue in its own vhost: TCP connection between publisher/consumer and broker. Long connection
Channel : If a Connection is established every time RabbitMQ is accessed, the overhead of establishing a TCP Connection when the message volume is large will be huge and the efficiency will be low. Channel is a logical connection established inside the connection. If the application supports multi-threading, usually each thread creates a separate channel for communication. AMQP method includes the channel id to help the client and message broker identify the channel, so the channels are completely isolated of. As a lightweight Connection, Channel greatly reduces the overhead of establishing a TCP connection by the operating system
Exchange: The message arrives at the first stop of the broker, according to the distribution rules, matches the routing key in the query table, and distributes the message to the queue. Commonly used types are: direct (point-to-point), topic (publish-subscribe) and fanout (multicast)
Queue : the message is finally sent here to wait for the consumer to take it away. Stored in the queue
Binding : between exchange and queue Virtual connection, the binding can contain routing key. Binding information is stored in the query table in the exchange for the basis of message distribution
2. Use steps
2.1 Environment configuration
Download the RabbitMQ image on a virtual machine installed with docker
Execute the following command
docker run -d --name rabbit -e RABBITMQ_DEFAULT_USER=admin -e RABBITMQ_DEFAULT_PASS=admin -p 15672:15672 -p 5672:5672 -p 25672:25672 -p 61613:61613 -p 1883:1883 rabbitmq:management
This command will automatically pull the image and start the container in the background
Then enter http://virtual machine address ip:15672/#/ on the browser to access the visual interface of rabbitmq
2.2 Test using --java
Create two new maven projects, one is the producer and the other is the consumer
Introduce dependencies in each
<!--rabbitmq connection dependencies --> <dependency> <groupId>com.rabbitmq</groupId> <artifactId>amqp-client</artifactId> <version>5.16.0</version> </dependency>
2.2.1 Simple Mode
producer side code
/**
* 测试简单模式
* @throws Exception
*/
@Test
public void testSimple() throws Exception{
//创建连接工厂对象---配置连接信息
ConnectionFactory factory=new ConnectionFactory();
factory.setUsername("admin");
factory.setPassword("admin");
factory.setVirtualHost("/");
factory.setHost("192.168.223.158");
factory.setPort(5672);
//创建连接对象Connection
Connection connection = factory.newConnection();
//创建channel信道
Channel channel=connection.createChannel();
//创建队列
/**
* String queue,队列名称
* boolean durable, 是否持久化
* boolean exclusive,是否私有
* boolean autoDelete,是否自动删除
* Map<String, Object> arguments: 队列的参数
*/
channel.queueDeclare("simple_queue",true,false,false,null);
//发消息
/**
* String exchange, 简单模式没有交换机所以写为""
* String routingKey, 路由key-- 默认队列名
* BasicProperties props, 消息的属性
* byte[] body: 消息的内容
*/
String msg="想睡觉吗? 那就睡吧";
channel.basicPublish("","simple_queue",null,msg.getBytes());
channel.close();
connection.close();
}consumer side code
public class TestSimple {
public static void main(String[] args) throws Exception {
//创建连接工厂对象---配置连接信息
ConnectionFactory factory=new ConnectionFactory();
factory.setUsername("admin");
factory.setPassword("admin");
factory.setVirtualHost("/");
factory.setHost("192.168.223.158");
factory.setPort(5672);
//创建连接对象Connection
Connection connection = factory.newConnection();
//创建channel信道
Channel channel=connection.createChannel();
//创建队列:如果存在指定的队列名 则不创建
/**
* String queue,队列名称
* boolean durable, 是否持久化
* boolean exclusive,是否私有
* boolean autoDelete,是否自动删除
* Map<String, Object> arguments: 队列的参数
*/
channel.queueDeclare("simple_queue",true,false,false,null);
//获取消息
/**
* (String queue,队列名
* boolean autoAck, 自动确认
* Consumer callback : 回调函数.
*/
Consumer consumer = new DefaultConsumer(channel){
@Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
//body就是从队列中获取的消息内容
String msg = new String(body);
System.out.println("接受的消息内容:"+msg);
//根据消息的内容写自己的业务代码
}
};
channel.basicConsume("simple_queue",true,consumer);
}
}
A simple pattern producer sends messages to a queue. Then the consumer directly gets the message and processes it
2.2.2 Work queues work queue mode
If there are multiple consumers in a queue, the relationship between consumers for the same message is a competitive relationship.
Work Queues The use of work queues can improve the speed of task processing when tasks are too heavy or there are many tasks. For example, if there are multiple SMS service deployments, only one node needs to send them successfully.
producer code
/**
* 工作模式
* @throws Exception
*/
@Test
public void testWorker() throws Exception{
//创建连接工厂对象---配置连接信息
ConnectionFactory factory=new ConnectionFactory();
factory.setUsername("admin");
factory.setPassword("admin");
factory.setVirtualHost("/");
factory.setHost("192.168.223.158");
factory.setPort(5672);
//创建连接对象Connection
Connection connection = factory.newConnection();
//创建channel信道
Channel channel=connection.createChannel();
//创建队列
/**
* String queue,队列名称
* boolean durable, 是否持久化
* boolean exclusive,是否私有
* boolean autoDelete,是否自动删除
* Map<String, Object> arguments: 队列的参数
*/
channel.queueDeclare("worker_queue",true,false,false,null);
//发消息
/**
* String exchange, 简单模式没有交换机所以写为""
* String routingKey, 路由key-- 默认队列名
* BasicProperties props, 消息的属性
* byte[] body: 消息的内容
*/
for(int i=0;i<20;i++) {
String msg = "想睡觉吗? 忍忍就过去了----->"+i;
channel.basicPublish("", "worker_queue", null, msg.getBytes());
}
channel.close();
connection.close();
}
two consumer codes
public class TestWorker01 {
public static void main(String[] args) throws Exception {
//创建连接工厂对象---配置连接信息
ConnectionFactory factory=new ConnectionFactory();
factory.setUsername("admin");
factory.setPassword("admin");
factory.setVirtualHost("/");
factory.setHost("192.168.223.158");
factory.setPort(5672);
//创建连接对象Connection
Connection connection = factory.newConnection();
//创建channel信道
final Channel channel=connection.createChannel();
//每次处理一条消息
channel.basicQos(1);
//创建队列:如果存在指定的队列名 则不创建
/**
* String queue,队列名称
* boolean durable, 是否持久化
* boolean exclusive,是否私有
* boolean autoDelete,是否自动删除
* Map<String, Object> arguments: 队列的参数
*/
channel.queueDeclare("worker_queue",true,false,false,null);
//获取消息
/**
* (String queue,队列名
* boolean autoAck, 自动确认
* Consumer callback : 回调函数.
*/
Consumer consumer = new DefaultConsumer(channel){
@Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
try {
Thread.sleep(200);
} catch (InterruptedException e) {
e.printStackTrace();
}
//body就是从队列中获取的消息内容
String msg = new String(body);
System.out.println("接受的消息内容:"+msg);
//根据消息的内容写自己的业务代码
//long deliveryTag, boolean multiple
//手动告诉队列消息处理完毕
channel.basicAck(envelope.getDeliveryTag(),false);
}
};
channel.basicConsume("worker_queue",false,consumer);
}
}
public class TestWorker02 {
public static void main(String[] args) throws Exception {
//创建连接工厂对象---配置连接信息
ConnectionFactory factory=new ConnectionFactory();
factory.setUsername("admin");
factory.setPassword("admin");
factory.setVirtualHost("/");
factory.setHost("192.168.223.158");
factory.setPort(5672);
//创建连接对象Connection
Connection connection = factory.newConnection();
//创建channel信道
final Channel channel=connection.createChannel();
//每次处理一条消息
channel.basicQos(1);
//创建队列:如果存在指定的队列名 则不创建
/**
* String queue,队列名称
* boolean durable, 是否持久化
* boolean exclusive,是否私有
* boolean autoDelete,是否自动删除
* Map<String, Object> arguments: 队列的参数
*/
channel.queueDeclare("worker_queue",true,false,false,null);
//获取消息
/**
* (String queue,队列名
* boolean autoAck, 自动确认
* Consumer callback : 回调函数.
*/
Consumer consumer = new DefaultConsumer(channel){
@Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
//body就是从队列中获取的消息内容
String msg = new String(body);
System.out.println("接受的消息内容:"+msg);
//根据消息的内容写自己的业务代码
//long deliveryTag, boolean multiple
//手动告诉队列消息处理完毕
channel.basicAck(envelope.getDeliveryTag(),false);
}
};
channel.basicConsume("worker_queue",false,consumer);
}
}
If one of the consumers has a slow processing time, the other will consume all the messages.
2.2.3 Pub/Sub subscription model
In the subscription model, there is an additional Exchange role, and the process has changed slightly:
P: Producer, that is, the program that wants to send messages, but it is no longer sent to the queue, but to X (exchange)
C: Consumer , the receiver of the message will always wait for the message to arrive
Queue: message queue, receive messages, cache messages
Exchange: switch (X). On the one hand, receive messages sent by producers. On the other hand, knowing how to process the message, such as delivering it to a particular queue, delivering it to all queues, or discarding the message. How it works depends on the type of Exchange. Exchange has the following three common types:
Fanout: Broadcast, deliver the message to all queues bound to the exchange
Direct: Directed, deliver the message to the queue that matches the specified routing key
Topic: Wildcard, deliver the message to the queue that matches the routing pattern (routing pattern) Mode) queue
Exchange (exchange) is only responsible for forwarding messages, and does not have the ability to store messages, so if there is no queue bound to Exchange, or there is no queue that meets the routing rules, then the message will be lost!
producer code
/**
* 发布订阅模式
* @throws Exception
*/
@Test
public void testPublisher() throws Exception{
//创建连接工厂对象---配置连接信息
ConnectionFactory factory=new ConnectionFactory();
factory.setUsername("admin");
factory.setPassword("admin");
factory.setVirtualHost("/");
factory.setHost("192.168.223.158");
factory.setPort(5672);
//创建连接对象Connection
Connection connection = factory.newConnection();
//创建channel信道
Channel channel=connection.createChannel();
//创建交换机
/**
* String exchange, 交换机的名称
* BuiltinExchangeType type, 交换机的类型
* boolean durable: 是否持久化
*/
channel.exchangeDeclare("fanout_exchange",BuiltinExchangeType.FANOUT,true);
//创建队列
channel.queueDeclare("fanout_queue01",true,false,false,null);
channel.queueDeclare("fanout_queue02",true,false,false,null);
//队列交换机绑定
channel.queueBind("fanout_queue01","fanout_exchange","");
channel.queueBind("fanout_queue02","fanout_exchange","");
String msg="想睡觉吗? 忍忍就过去了????";
channel.basicPublish("fanout_exchange","",null,msg.getBytes());
}
consumer's code
public class TestPublisher01 {
public static void main(String[] args) throws Exception {
//创建连接工厂对象---配置连接信息
ConnectionFactory factory=new ConnectionFactory();
factory.setUsername("admin");
factory.setPassword("admin");
factory.setVirtualHost("/");
factory.setHost("192.168.223.158");
factory.setPort(5672);
//创建连接对象Connection
Connection connection = factory.newConnection();
//创建channel信道
final Channel channel=connection.createChannel();
//每次处理一条消息
channel.basicQos(1);
//获取消息
/**
* (String queue,队列名
* boolean autoAck, 自动确认
* Consumer callback : 回调函数.
*/
Consumer consumer = new DefaultConsumer(channel){
@Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
try {
Thread.sleep(200);
} catch (InterruptedException e) {
e.printStackTrace();
}
//body就是从队列中获取的消息内容
String msg = new String(body);
System.out.println("接受的消息内容:"+msg);
//根据消息的内容写自己的业务代码
//long deliveryTag, boolean multiple
//手动告诉队列消息处理完毕
channel.basicAck(envelope.getDeliveryTag(),false);
}
};
channel.basicConsume("fanout_queue01",false,consumer);
}
}public class TestPublisher02 {
public static void main(String[] args) throws Exception {
//创建连接工厂对象---配置连接信息
ConnectionFactory factory=new ConnectionFactory();
factory.setUsername("admin");
factory.setPassword("admin");
factory.setVirtualHost("/");
factory.setHost("192.168.223.158");
factory.setPort(5672);
//创建连接对象Connection
Connection connection = factory.newConnection();
//创建channel信道
final Channel channel=connection.createChannel();
//每次处理一条消息
channel.basicQos(1);
//获取消息
/**
* (String queue,队列名
* boolean autoAck, 自动确认
* Consumer callback : 回调函数.
*/
Consumer consumer = new DefaultConsumer(channel){
@Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
try {
Thread.sleep(200);
} catch (InterruptedException e) {
e.printStackTrace();
}
//body就是从队列中获取的消息内容
String msg = new String(body);
System.out.println("接受的消息内容:"+msg);
//根据消息的内容写自己的业务代码
//long deliveryTag, boolean multiple
//手动告诉队列消息处理完毕
channel.basicAck(envelope.getDeliveryTag(),false);
}
};
channel.basicConsume("fanout_queue02",false,consumer);
}
}
2.2.4 Routing routing mode
The binding between the queue and the exchange cannot be arbitrary, but must specify a RoutingKey (routing key).
When sending a message to Exchange, the sender must also specify the RoutingKey of the message.
Exchange will no longer hand over the message to each The bound queue is judged according to the Routing Key of the message. Only when the Routing Key of the queue is exactly the same as the Routing Key of the message, the message will be received
. key
X: Exchange (exchange), receive the message from the producer, and then deliver the message to the queue that exactly matches the routing key
C1: the consumer, where the queue specifies the message that needs the routing key to be error
C2: the consumer, where it is The queue specifies the message
producer code that requires the routing key to be info, error, and warning
@Test
public void testRouter() throws Exception{
//创建连接工厂对象---配置连接信息
ConnectionFactory factory=new ConnectionFactory();
factory.setUsername("admin");
factory.setPassword("admin");
factory.setVirtualHost("/");
factory.setHost("192.168.223.158");
factory.setPort(5672);
//创建连接对象Connection
Connection connection = factory.newConnection();
//创建channel信道
Channel channel=connection.createChannel();
//创建交换机
/**
* String exchange, 交换机的名称
* BuiltinExchangeType type, 交换机的类型
* boolean durable: 是否持久化
*/
channel.exchangeDeclare("router_exchange",BuiltinExchangeType.DIRECT,true);
//创建队列
channel.queueDeclare("router_queue01",true,false,false,null);
channel.queueDeclare("router_queue02",true,false,false,null);
//队列交换机绑定
channel.queueBind("router_queue01","router_exchange","error");
channel.queueBind("router_queue02","router_exchange","error");
channel.queueBind("router_queue02","router_exchange","info");
channel.queueBind("router_queue02","router_exchange","warning");
String msg="这都是你的警告---warning";
channel.basicPublish("router_exchange","warning",null,msg.getBytes());
}
Routing mode requires the queue to specify the routing key when binding the switch , and the message will be forwarded to the queue that matches the routing key.
Summarize
To be added