1. Related concepts of MQ
1.1. What is MQ?
MQ (message queue), literally, is a queue, FIFO first in, first out, but the content stored in the queue is just messages. It is also a cross-process communication mechanism for upstream and downstream message transmission. In the Internet architecture, MQ is a very common upstream and downstream "logical decoupling + physical decoupling" message communication service. After using MQ, the message sending upstream only needs to rely on MQ, not on other services.
1.2. Why use MQ?
1. Traffic peak elimination
For example, if the order system can handle up to 10,000 orders, this processing capacity is more than enough to handle orders during normal periods. During normal periods, we can return results in one second after placing orders. However, during the peak period, if there are 20,000 orders placed, the operating system cannot handle it. It can only limit the number of orders exceeding 10,000 and not allow users to place orders. Using the message queue as buffer, we can cancel this restriction and spread the orders placed within one second into a period of time for processing. At this time, some users may not receive the successful order operation more than ten seconds after placing the order, but it is better than not being able to place the order. Single experience is better.
2. Application decoupling
Take e-commerce applications as an example. The applications include order systems, inventory systems, logistics systems, and payment systems. After the user creates an order, if the inventory system, logistics system, and payment system are coupled and called, if any subsystem fails, the order operation will be abnormal. When converted to a message queue-based approach, the problems of inter-system calls will be greatly reduced. For example, a logistics system will take a few minutes to repair due to a failure. During these few minutes, the memory to be processed by the logistics system is cached in the message queue, and the user's order operation can be completed normally. When the logistics system is restored, just continue to process the order information. The mid-order user will not feel the failure of the logistics system, improving the availability of the system.
3. Asynchronous processing
Some calls between services are asynchronous. For example, A calls B, and B takes a long time to execute, but A needs to know when B can be completed. In the past, there were generally two ways. A would call B's query api query after a period of time. . Or A provides a callback api, and B calls the api to notify A service after execution. Both of these methods are not very elegant. Using the message bus can easily solve this problem. After A calls the B service, it only needs to monitor the message that B has completed. When B completes the processing, it will send a message to MQ, and MQ will Forward this message to service A. In this way, service A does not need to call B's query api in a loop, nor does it need to provide a callback api. Similarly, service B does not need to perform these operations. Service A can also get the message of successful asynchronous processing in a timely manner.
Common MQ products
ActiveMQ: Based on JMS RabbitMQ: Based on AMQP protocol, developed in erlang language, good stability
RocketMQ: Based on JMS, Alibaba product, currently handed over to the Apache Foundation
Kafka: Distributed messaging system, high throughput
RabbitMQ Quick Start
RabbitMQ is Message queue developed by Erlang language and implemented based on AMQP (Advanced Message Queue Advanced Message Queuing Protocol) protocol. It is a communication method between applications. Message queue is widely used in distributed system development. RabbitMQ official address: http://www.rabbitmq.com
Download and install
RabbitMQ is developed by the Erlang language. It is necessary to install the Erlang language environment corresponding to the RabbitMQ version. I will not explain the specifics. Search for tutorials by yourself. RabbitMQ official website download address: http://www.rabbitmq.com/download.html
1.3. Working principle of RabbitMQ
The following figure is the basic structure of RabbitMQ:
1. Message
. The message is not specific. It consists of a message header and a message body. The message body is opaque, and the message header consists of a series of optional attributes,
including routing-key (routing key), priority (priority relative to other messages), delivery-mode (indicating that the message may require persistent storage), etc.
2.
The producer of Publisher messages is also a client application that publishes messages to the exchange.
3. Exchange
exchanger, used to receive messages sent by producers and route these messages to queues in the server. .
4 Binding
binding, used for the association between the message queue and the exchanger. A binding is a routing rule that connects an exchanger and a message queue based on a routing key, so
the exchanger can be understood as a routing table composed of bindings.
5. Queue
message queue, used to save messages until they are sent to consumers. It is the container of messages and the end point of messages. A message can be put into one or more queues.
The message has been in the queue, waiting for the consumer to connect to the queue to pick it up.
If there are multiple consumers listening to the same queue at the same time, they will use polling to consume the messages in the queue. If someone exits in the middle, the remaining consumers will consume the messages in the queue.
Create connection factory
public static ConnectionFactory getFactory(){
// 创建连接工厂
ConnectionFactory factory = new ConnectionFactory();
factory.setUsername("root");
//tcp连接端口
factory.setPort(5672);
factory.setPassword("root");
factory.setHost("192.168.204.129");
return factory;
}
-
Create a connection (send directly without using a switch)
Producer:
private static void getConnectionFactory(){ ConnectionFactory factory = FactoryUtil.getFactory(); try { // 创建连接 Connection conn = factory.newConnection(); // 获得信道 Channel channel = conn.createChannel(); // 声明队列 channel.queueDeclare("myQueue",true,false,false,null); String message = "hello,rabbitmq....."+new Date(); // 发送消息到指定队列(交换机，路由键（队列名），属性，消息内容字节流) channel.basicPublish("","myQueue",null,message.getBytes()); System.out.println("消息已经发送"+new Date()); channel.close(); conn.close(); } catch (IOException e) { e.printStackTrace(); } catch (TimeoutException e) { e.printStackTrace(); } }consumer
public static void getReciver(){ // 连接工厂对象 ConnectionFactory factory = FactoryUtil.getFactory(); //创建连接 try { Connection conn = factory.newConnection(); // 通道 Channel channel = conn.createChannel(); // 声明队列 channel.queueDeclare("myQueue",true,false,false,null); // 消费消息--队列名，自动确认，消费者 channel.basicConsume("myQueue", true, new DefaultConsumer(channel) { @Override public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException { System.out.println("接受到消息："+new String(body,"utf-8")); } }); } catch (IOException e) { e.printStackTrace(); } catch (TimeoutException e) { e.printStackTrace(); } }
Exchange - Create a connection using a switch
You can also use this constant to declare the sending mode -BuiltinExchangeType
-
Direct mode
- ** Handle routing keys. **Requires a queue to be bound to an exchange,** requires that the message exactly match a specific routing key. **This is a complete match. If a queue is bound to the switch and requires the routing key "dog", only messages marked as "dog" will be forwarded. Dog.puppy will not be forwarded, nor dog.guard will be forwarded, only dog will be forwarded.
Producer:
/** *@Description//TODO Direct模型 *@Date2022/8/8 16:29 **/ private static void getSender(){ ConnectionFactory factory = FactoryUtil.getFactory(); try { Connection conn = factory.newConnection(); Channel channel = conn.createChannel(); //声明交换机--交换机名称，类型，持久化 channel.exchangeDeclare("directExchange","direct",true); // 发送消息 String message = "hello...direct exchagne..."+new Date(); // 交换机，指定的路由键，属性，消息内容字节流 channel.basicPublish("directExchange",**"green"**,null,message.getBytes()); System.out.println("消息发送成功...."); channel.close(); conn.close(); } catch (IOException e) { e.printStackTrace(); } catch (TimeoutException e) { e.printStackTrace(); } }consumer:
public static void getReciver(){ // 连接工厂对象 ConnectionFactory factory = FactoryUtil.getFactory(); //创建连接 try { Connection conn = factory.newConnection(); // 通道 Channel channel = conn.createChannel(); // 声明队列 channel.queueDeclare("direqueue1",true,false,false,null); // 声明交换机（交换机名，交换类型(需要全小写)，是否声明为持久层） channel.exchangeDeclare("directExchange",***"direct"***,true); // 绑定交接机(队列，交换机，用于绑定的路由键（子会接收路由键相同的消息）) channel.queueBind("direqueue1","directExchange",**"green"**); // 消费消息--队列名，自动确认，消费者 channel.basicConsume("direqueue1", true, new DefaultConsumer(channel) { @Override public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException { System.out.println("DirectReciver1接收到消息:"+new String(body,"utf-8")); } }); } catch (IOException e) { e.printStackTrace(); } catch (TimeoutException e) { e.printStackTrace(); } } -
broadcast mode-fanout
-
Routing keys are not processed . You simply bind the queue to the switch. A message sent to a switch will be forwarded to all queues bound to the switch . Much like a subnet broadcast, each host in the subnet gets a copy of the message. Fanout switch forwards messages the fastest
Producer:
/** *@Description//TODO Fanout模型-会发送给交换机中的所以队列 *@Date2022/8/8 16:29 **/ private static void getSender(){ ConnectionFactory factory = FactoryUtil.getFactory(); try { Connection conn = factory.newConnection(); Channel channel = conn.createChannel(); //声明交换机--交换机名称，类型，持久化 channel.exchangeDeclare("fanoutExchange","fanout",true); // 发送消息 String message = "hello...fanout exchagne..."+new Date(); // 交换机，不用指定路由键，属性，消息内容字节流 channel.basicPublish("fanoutExchange","",null,message.getBytes()); System.out.println("fanoutExchange消息发送成功...."); channel.close(); conn.close(); } catch (IOException e) { e.printStackTrace(); } catch (TimeoutException e) { e.printStackTrace(); } }consumer:
public static void getReciver(){ // 连接工厂对象 ConnectionFactory factory = FactoryUtil.getFactory(); //创建连接 try { Connection conn = factory.newConnection(); // 通道 Channel channel = conn.createChannel(); // 声明队列 channel.queueDeclare("fanout1",true,false,false,null); // 声明交换机 channel.exchangeDeclare("fanoutExchange","fanout",true); // 绑定交接机(队列，交换机，用于绑定的路由键) channel.queueBind("fanout1","fanoutExchange","green"); // 消费消息--队列名，自动确认，消费者 channel.basicConsume("fanout1", true, new DefaultConsumer(channel) { @Override public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException { System.out.println("FanoutReciver1接收到消息:"+new String(body,"utf-8")); } }); } catch (IOException e) { e.printStackTrace(); } catch (TimeoutException e) { e.printStackTrace(); } } -
-
topic mode
-
The producer sends the data to the switch and sets a specific routing key; the consumer receives the messages in the queue that meet the conditions according to the wildcard character of the queue.
- : represents 1 word
#: represents 0 to more
Producer:
/** *@Description//TODO Topic模型- *@Date2022/8/8 16:29 **/ private static void getSender(){ ConnectionFactory factory = FactoryUtil.getFactory(); try { Connection conn = factory.newConnection(); Channel channel = conn.createChannel(); //声明交换机--交换机名称，类型，持久化 channel.exchangeDeclare("topicExchange", BuiltinExchangeType.TOPIC,true); // 发送消息 String message = "hello...fanout exchagne..."+new Date(); // 交换机，不用指定路由键，属性，消息内容字节流 channel.basicPublish("topicExchange","aa.bb.zez",null,message.getBytes()); System.out.println("fanoutExchange消息发送成功...."); channel.close(); conn.close(); } catch (IOException e) { e.printStackTrace(); } catch (TimeoutException e) { e.printStackTrace(); } }consumer:
public static void getReciver(){ // 连接工厂对象 ConnectionFactory factory = FactoryUtil.getFactory(); //创建连接 try { Connection conn = factory.newConnection(); // 通道 Channel channel = conn.createChannel(); // 声明队列 channel.queueDeclare("topic3",true,false,false,null); // 声明交换机 channel.exchangeDeclare("topicExchange", BuiltinExchangeType.TOPIC,true); // 绑定交接机(队列，交换机，用于绑定的路由键) channel.queueBind("topic3","topicExchange","*.bb.#"); // 消费消息--队列名，自动确认，消费者 channel.basicConsume("topic3", true, new DefaultConsumer(channel) { @Override public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException { System.out.println("topicExchange3接收到消息:"+new String(body,"utf-8")); } }); } catch (IOException e) { e.printStackTrace(); } catch (TimeoutException e) { e.printStackTrace(); } -
Transaction:
When a producer sends a message, a transaction can be used to ensure the atomicity of the sending.
- Start a transaction:
channel.txSelect(); - Submit transaction:
channel.txCommit();
Enable listening confirmation:
Sync mode:
- Turn on monitoring confirmation mode:
channel.confirmSelect(); - Execute monitoring confirmation:
channel.waitForConfirmsOrDie();
Asynchronous mode:
This mode will perform monitoring in an asynchronous manner to determine which ones succeed and which ones fail. If they fail, they will be re-executed and sent.
/**
*@Description//TODO currentTimeMillis启用发布者确认使用异步执行，该
*@Date2022/8/8 16:29
**/
private static void getSender(){
ConnectionFactory factory = FactoryUtil.getFactory();
try {
Connection conn = factory.newConnection();
Channel channel = conn.createChannel();
//声明交换机--交换机名称，类型，持久化
channel.exchangeDeclare("transExchange", BuiltinExchangeType.DIRECT,true);
// 启用发布者确认模式
channel.confirmSelect();
long l = System.currentTimeMillis();
for (int i = 0; i < 1000; i++) {
// 发送消息
String message1 = "hello...direct exchagne1..."+i;
// 交换机，指定的路由键，属性，消息内容字节流
channel.basicPublish("transExchange","trans",null,message1.getBytes());
}
// 执行监听确认
channel.addConfirmListener(new ConfirmListener() {
// 确认消息
@Override
public void handleAck(long deliveryTag, boolean multiple) throws IOException {
System.out.println("确认消息****："+deliveryTag+",状态："+multiple);
}
// 未确认
@Override
public void handleNack(long deliveryTag, boolean multiple) throws IOException {
System.err.println("未消息####："+deliveryTag+",状态："+multiple);
}
});
long j = System.currentTimeMillis();
System.out.println("消息使用时间---"+(j-l));
// channel.close();
// conn.close();
} catch (Exception e) {
e.printStackTrace();
}
}