RabbitMQ exchange exchange mechanism

Others 2022-05-26 17:03:22 views: 0

content

  • RabbitMQ Concepts
  • exchange switch mechanism
    • What is a switch
    • binding？
    • Direct Exchange Switch
    • Topic Exchange switch
    • Fanout Exchange
    • Header Exchange
  • Hello - Demo of RabbitMQ (springboot implementation)
  • Hello Demo of RabbitMQ (spring xml implementation)
  • RabbitMQ application and problems in production environment
    • Spring RabbitMQ annotations
    • JSON transfer of messages
    • Message persistence, disconnection and reconnection, ACK.

RabbitMQ Concepts

RabbitMQ is a message queue, which is mainly used to realize the asynchrony and decoupling of applications, and can also play the role of message buffering and message distribution. RabbitMQ uses the AMQP protocol, which is a binary protocol. Port 5672 is started by default.

In RabbitMQ, the structure is as follows:


rabbitmq
  • The P on the left represents the producer, which is the program that sends messages to RabbitMQ.
  • In the middle is RabbitMQ, which includes exchanges and queues.
  • The C on the right represents the consumer, which is the program that gets messages from RabbitMQ.

Well, there are 4 more important concepts, namely: virtual host, switch, queue, and binding.

  • Virtual Host: A virtual host holds a set of switches, queues and bindings. Why do you need multiple virtual hosts? Quite simply, in RabbitMQ, users can only control permissions at the granularity of virtual hosts. Therefore, if group A needs to be disabled from accessing the switches/queues/bindings of group B, a virtual host must be created for A and B respectively. Every RabbitMQ server has a default virtual host "/".
  • Exchange: Exchange is used to forward messages, but it will not be stored . If there is no Queue bind to Exchange, it will directly discard the messages sent by the Producer.
    • There is a more important concept here: routing keys . When the message arrives at the switch, the interaction will be forwarded to the corresponding queue, so which queue is forwarded to depends on the routing key.
  • Binding: That is, the switch needs to be bound to the queue, which is a many-to-many relationship as shown in the figure above.

exchange switch mechanism

What is a switch

The message model of rabbitmq actually does not send messages directly to the queue. There is an exchange in the middle for message distribution, and the producer does not even know which queue to send the message to. Therefore, when an exchange receives a message, it must know exactly how to distribute it. Is it appending to a queue of certain rules, or appending to multiple queues, or is it discarded? These rules are defined by the four types of exchagne.

The core idea in the messaging model in RabbitMQ is that the producer never sends any messages directly to a queue. Actually, quite often the producer doesn't even know if a message will be delivered to any queue at all.

Instead, the producer can only send messages to an exchange. An exchange is a very simple thing. On one side it receives messages from producers and the other side it pushes them to queues. The exchange must know exactly what to do with a message it receives. Should it be appended to a particular queue? Should it be appended to many queues? Or should it get discarded. The rules for that are defined by the exchange type.

exchange is a message agent, defined in each virtual host. Its responsibility is to route messages to different queues.

binding？

Exchange and queue are associated through routing-key, and the relationship between the two is binding. As shown in the figure below, X represents the switch, and red represents the queue. The switch uses a routing-key to bind a queue. What does the routing-key do? See Direct exchange type switches.

Directed Exchange

The routing key exchange, after receiving the message, the exchange will send the message to the queue of the specified routing-key. How does the message exchange know? In fact, when the producer delivers the message, it will add the routing-key to the message header. routing-key is just an attribute of messagegae.

A direct exchange delivers messages to queues based on a message routing key. The routing key is a message attribute added into the message header by the producer. The routing key can be seen as an "address" that the exchange use to decide how to route the message. A message goes to the queue(s) whose binding key exactly matches the routing key of the message.

Default Exchange
is a special Direct Exchange, which is a default exchange within rabbitmq. The name of the switch is an empty string, and all queues are bound to this switch by default. For all queues bound to this switch, the routing-key is the same as the name of the queue.

Topic Exchange

Wildcard exchange, the exchange will send messages to one or more queues that satisfy the wildcard rules routing-key. Among them, * table number matches one word, # matches multiple words and paths, and the paths are separated by . For example, the routing-key that satisfies a.*.c has a.hello.c; the routing-key that satisfies #.hello has abchelo.

Fanout Exchange

A sector switch that sends messages to all queues bound to the switch. This is the publisher/subscribe mode. Best for broadcasting.
All routing-keys specified on the exchagne will be ignored.

The fanout copies and routes a received message to all queues that are bound to it regardless of routing keys or pattern matching as with direct and topic exchanges. Keys provided will simply be ignored.

Header Exchange

Set the switch of the header attribute parameter type.

RabbitMQ 的 Hello Demo

Not to mention the installation, it is recommended to follow the official documentation. Paste the code first, explain later, the code is as follows:

Configure switches, queues, bindings between switches and queues, and message monitoring containers:

@Configuration
@Data
public class RabbitMQConfig {

    final static String queueName = "spring-boot";

    @Bean
    Queue queue() {
        return new Queue(queueName, false);
    }

    @Bean
    TopicExchange exchange() {
        return new TopicExchange("spring-boot-exchange");
    }

    @Bean
    Binding binding(Queue queue, TopicExchange exchange) {
        return BindingBuilder.bind(queue).to(exchange).with(queueName);
    }

    @Bean
    SimpleMessageListenerContainer container(ConnectionFactory connectionFactory, MessageListenerAdapter listenerAdapter) {
        SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
        container.setConnectionFactory(connectionFactory);
        container.setQueueNames(queueName);
        container.setMessageListener(listenerAdapter);
        return container;
    }

    @Bean
    Receiver receiver() {
        return new Receiver();
    }
    @Bean
    MessageListenerAdapter listenerAdapter(Receiver receiver) {
        return new MessageListenerAdapter(receiver, "receiveMessage");
    }
}

Configure the recipient (ie, the consumer):

public class Receiver {

    private CountDownLatch latch = new CountDownLatch(1);

    public void receiveMessage(String message) {
        System.out.println("Received <" + message + ">");
        latch.countDown();
    }

    public CountDownLatch getLatch() {
        return latch;
    }
}

Configure the sender (ie the producer):

@RestController
public class Test {
    @Autowired
    RabbitTemplate rabbitTemplate;

    @RequestMapping(value = "/test/{abc}",method = RequestMethod.GET)
    public String test(@PathVariable(value = "abc") String abc){
        rabbitTemplate.convertAndSend("spring-boot", abc + " from RabbitMQ!");
        return  "abc";
    }
}

The above can realize a simple RabbitMQ Demo, the specific code is: click here

So, here, it is divided into three parts for analysis: sending messages, switching queues, and receiving messages.

  • For sending messages: We can generally use RabbitTemplate, which is encapsulated by Spring for us to send information, and we rabbitTemplate.convertAndSend("spring-boot", xxx);can .
  • For switch queues: As in the code above, we need to configure switches TopicExchange, configure queues Queue, and configure bindings between themBinding
  • For receiving messages: First, we need to create a message listening container, and then register our receivers in the container. In this way, if there is information in the queue, the corresponding methods of the receivers will be called. container.setMessageListener(listenerAdapter);As shown in the above code , MessageListenerAdapter can be regarded as a wrapper class of our receiver, new MessageListenerAdapter(receiver, "receiveMessage");indicating which method of the receiver to call if there is a message.

Hello Demo of RabbitMQ (spring xml implementation)

The implementation of RabbitMQ in spring xml is simple, readable, and simple to configure. The configuration and implementation are as follows.

The configuration of rabbitmq has been described above. The xml method stores user configuration information through the properties file:

mq.host=127.0.0.1
mq.username=guest
mq.password=guest
mq.port=5672

Configure the application-mq.xml configuration file to declare connections, switches, queues, and consumer listeners.

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:rabbit="http://www.springframework.org/schema/rabbit"
       xmlns:context="http://www.springframework.org/schema/context"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
    http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
     http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.0.xsd
    http://www.springframework.org/schema/rabbit
    http://www.springframework.org/schema/rabbit/spring-rabbit-1.0.xsd" >
    <description>rabbitmq 连接服务配置</description>

    <!-- 连接配置 -->
    <context:property-placeholder location="classpath:mq.properties" />
    <rabbit:connection-factory id="connectionFactory" host="${mq.host}" username="${mq.username}" password="${mq.password}" port="${mq.port}"/>
    <rabbit:admin connection-factory="connectionFactory"/>
    <!-- spring template声明-->
    <rabbit:template exchange="amqpExchange" id="amqpTemplate"  connection-factory="connectionFactory" />

    <!--申明queue-->
    <rabbit:queue id="test_queue_key" name="test_queue_key" durable="true" auto-delete="false" exclusive="false" />
    <!--申明exchange交换机并绑定queue-->
    <rabbit:direct-exchange name="amqpExchange" durable="true" auto-delete="false" id="amqpExchange">
        <rabbit:bindings>
            <rabbit:binding queue="test_queue_key" key="test_queue_key"/>
        </rabbit:bindings>
    </rabbit:direct-exchange>


    <!--consumer配置监听-->
    <bean id="reveiver" class="com.demo.mq.receive.Reveiver" />
    <rabbit:listener-container connection-factory="connectionFactory" acknowledge="auto">
        <rabbit:listener queues="test_queue_key" ref="reveiver" method="receiveMessage"/>
    </rabbit:listener-container>
</beans>

In the above code, there is not much to say about the introduction of the properties file.

<rabbit:connection-factory>The tag declares the factory that creates the connection.

<rabbit-template>Declare spring template, the same as using template in spring above. template can declare exchange.

<rabbit:queue>Declare a queue and set the configuration items of the queue. You can understand the configuration items of the queue directly by looking at the tag attributes.

<rabbit:direct-exchange>Declare the switch and bind the queue.

<rabbit:listener-container>Declare the monitor container and configure the consumer and monitor routing-key.

The rest is simple, import the rabbitmq configuration in application-context.xml.

<?xml version="1.0" encoding="UTF-8"?>
<beans
        xmlns="http://www.springframework.org/schema/beans"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xmlns:task="http://www.springframework.org/schema/task"
        xmlns:context="http://www.springframework.org/schema/context"
        xmlns:aop="http://www.springframework.org/schema/aop"
        xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
       http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.0.xsd
       http://www.springframework.org/schema/task http://www.springframework.org/schema/task/spring-task-3.0.xsd
       http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-3.0.xsd">

    <context:component-scan base-package="com.demo.**" />
    <import resource="application-mq.xml" />
</beans>

Producer implementation, send messages or use template's convertAndSend() to deliver messages.

@Service
public class Producer {

    @Autowired
    private AmqpTemplate amqpTemplate;

    private final static Logger logger = LoggerFactory.getLogger(Producer.class);

    public void sendDataToQueue(String queueKey, Object object) {
        try {
            amqpTemplate.convertAndSend(queueKey, object);
        } catch (Exception e) {
            e.printStackTrace();
            logger.error("exeception={}",e);
        }

    }
}

Configure the consumer

package com.demo.mq.receive;

import org.springframework.stereotype.Service;
import java.util.concurrent.CountDownLatch;

@Service
public class Reveiver {
    private CountDownLatch latch = new CountDownLatch(1);

    public void receiveMessage(String message) {
        System.out.println("reveice msg=" + message.toString());
        latch.countDown();
    }
}

Test deliver message

Controller
@RequestMapping("/demo/")
public class TestController {
    private final static Logger logger = LoggerFactory.getLogger(TestController.class);
    @Resource
    private Producer producer;


    @RequestMapping("/test/{msg}")
    public String send(@PathVariable("msg") String msg){
        logger.info("#TestController.send#abc={msg}", msg);
        System.out.println("msg="+msg);
        producer.sendDataToQueue("test_queue_key",msg);
        return "index";
    }
}

RabbitMQ application and problems in production environment

In a production environment, since Spring provides some convenient annotations for RabbitMQ, these annotations can be used first. E.g:

  • @EnableRabbit: @EnableRabbit and @Configuration annotations are used together in a class. If the class can return a bean of type RabbitListenerContainerFactory, it is equivalent to connecting the terminal (consumer) with RabbitMQ. Ps: (The generator is not connected to RabbitMQ through RabbitListenerContainerFactory, but through RabbitTemplate)
  • @RabbitListener: When there is a message in the corresponding queue, the method modified by this annotation will be executed.
  • @RabbitHandler: The receiver can listen to multiple queues, and the types of messages in different queues may be different. This annotation can make different messages respond to different methods.

For the specific use of these annotations, you can refer to the code here: Click here

First of all, RabbitMQ in the production environment may not be on the producer or consumer machine, so it is necessary to redefine the ConnectionFactory, namely:

@Bean
ConnectionFactory connectionFactory() {
    CachingConnectionFactory connectionFactory = new CachingConnectionFactory(host, port);
    connectionFactory.setUsername(userName);
    connectionFactory.setPassword(password);
    connectionFactory.setVirtualHost(vhost);
    return connectionFactory;
}

Here, you can reset the ip, port, virtual host, username, and password of RabbitMQ that needs to be connected.

Then, you can consider the production side first. The production side needs to connect to RabbitMQ, then it can be connected through RabbitTemplate. Ps: (RabbitTemplate is used by the production side to send messages to the switch), the following code:

@Bean(name="myTemplate")
RabbitTemplate rabbitTemplate(ConnectionFactory connectionFactory) {
    RabbitTemplate template = new RabbitTemplate(connectionFactory);
    template.setMessageConverter(integrationEventMessageConverter());
    template.setExchange(exchangeName);
    return template;
}

In this code, new RabbitTemplate(connectionFactory);the production end is set to connect to RabbitMQ, and template.setMessageConverter(integrationEventMessageConverter());the format of the message sent by the production end to the switch is set. In the integrationEventMessageConverter()code :

public MessageConverter integrationEventMessageConverter() {
    Jackson2JsonMessageConverter messageConverter = new Jackson2JsonMessageConverter();
    return messageConverter;
}

Jackson2JsonMessageConverterJSON is specified above . The end of the above code template.setExchange(exchangeName);specifies which exchange the producer should send the message to.

With the above, then we can use rabbitTemplate.convertAndSend("spring-boot", xxx);to send messages, xxx represents any type, because the above settings will help us convert these types into JSON transmission.

Next, the production side sends what we said, so now we can look at the consumer side:

For the consumer side, we can just create SimpleRabbitListenerContainerFactoryit, which can help us generate a RabbitListenerContainer, and then we use @RabbitListener to specify the method to be processed when the receiver receives the information.

@Bean(name="myListenContainer")
public SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory() {
    SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
    factory.setMessageConverter(integrationEventMessageConverter());
    factory.setConnectionFactory(connectionFactory());
    return factory;
}

This factory.setMessageConverter(integrationEventMessageConverter());specifies that when we accept the message, the message transmitted in JSON can be converted into the corresponding type and passed to the method. E.g:

@Slf4j
@Component
@RabbitListener(containerFactory = "helloRabbitListenerContainer",queues = "spring-boot")
public class Receiver {
    @RabbitHandler
    public void receiveTeacher(Teacher teacher) {
        log.info("##### = {}",teacher);
    }
}

Problems that may arise:

Endurance

In a production environment, we need to consider what if the producer hangs, the consumer hangs, or rabbitmq hangs. Generally speaking, if the producer hangs or the consumer hangs, it actually has no effect, because the message is in the queue. So in case rabbitmq hangs, what to do with the messages in the queue before, in fact, message persistence can be done, and RabbitMQ will save the information on the disk.

The approach is to first get a Channel object from the Connection object, and then set message persistence through the object.

The producer or consumer disconnects and reconnects

Here Spring has an automatic reconnection mechanism.

ACK confirmation mechanism

It may take a while for each Consumer to process the received data. If in this process, the Consumer makes an error, exits abnormally, and the data has not been processed, then unfortunately, this data is lost. Because we use the no-ack method for confirmation, that is, every time the Consumer receives the data, regardless of whether the processing is completed or not, the RabbitMQ Server will immediately mark the Message as complete, and then delete it from the queue.

If a Consumer exits abnormally, the data it processed can be processed by another Consumer, so that the data will not be lost in this case (note that this is the case).
In order to ensure that data is not lost, RabbitMQ supports a message confirmation mechanism, namely acknowledgments. In order to ensure that the data is processed correctly and not just received by the Consumer, then we cannot use no-ack. Instead, the ack should be sent after processing the data.

The ack sent after processing the data tells RabbitMQ that the data has been received, the processing is complete, and RabbitMQ can safely delete it.
If the Consumer exits but does not send an ack, RabbitMQ will send this Message to the next Consumer. This ensures that the data will not be lost if the Consumer exits abnormally.

Personal questions about RabbitMQ ACK, help: click here

Summarize

  1. RabbitMQ role: asynchronous, decoupling, buffering, message distribution.
  2. RabbitMQ is mainly divided into 3 parts, producers, exchanges and queues, and consumers.
  3. Need to pay attention to message persistence, the purpose is to prevent RabbitMQ from going down; consider the ACK mechanism, the purpose is how to deal with the subsequent if the consumer fails to process the message.

write at the end

  1. Write it out and speak it out before you know if it's right or wrong, and only if you know it's wrong can you correct it, and only when you correct it can you grow.
  2. In terms of technology, I hope everyone has no room for sand. If there is something wrong or something that needs to be improved, I hope you can point it out, thank you very much.



Text/Mooner_guo (author of Jianshu)
original link: http://www.jianshu.com/p/a5f7fce67803
The copyright belongs to the author, please contact the author for authorization and mark "author of Jianshu".

Guess you like

Origin http://10.200.1.11:23101/article/api/json?id=326995755&siteId=291194637
Recommended
Ranking
error: (-215:Assertion failed) !_img.empty() in function ‘cv::imwrite‘
Database migration between Navicat servers
Minimum number of rotation of the array: Array
balenaEtcher for mac (make a boot disk software) v1.5.67
Custom processing serialization and deserialization in jackson
Mu-en-mask system development software
Mastering Regular Expressions
Find mileage Java--
Web pages can not directly concern the public micro-channel number how to do? A key to arouse public concern number of micro-channel solutions
[CodeForces - 739B] Alyona and a tree Tree + [difference] + bipartite
Daily
More
2024-05-12(28)
2024-05-11(32)
2024-05-10(34)
2024-05-09(32)
2024-05-08(18)
2024-05-07(34)
2024-05-06(6)
2024-05-05(0)
2024-05-04(18)
2024-05-03(8)

Code World

© 2018 codetd.com