[JavaWeb microservice architecture project-Leyou Mall day11]-Use RabbitMQ to achieve data synchronization between search and static pages

0. Learning objectives

• Understand common MQ products
• Understand the 5 message models of RabbitMQ
• Use MQ to realize data synchronization between search and static pages

Source code and notes:
Link: https://pan.baidu.com/s/1REHmVFclFI9_pu1ESB_sZw
Extraction code: voyc

1.RabbitMQ

1.1. Search and commodity service issues

At present, we have completed the development of product details and search system. Let's think about it, is there a problem?

• The original data of the product is stored in the database, and all additions, deletions, changes and checks are completed in the database.
• The source of the search service data is the index database. If the database commodities change, the index database data cannot be updated in time.
• The product details have been staticized on the page, and the static page data will not change with the database products.

If we modify the price of the product in the background, the search page and product detail page still display the old price, which is obviously wrong. How to solve it?

There are two solutions:

• Solution 1: Whenever adding, deleting and modifying products in the background, the index database data and static pages should be modified at the same time
• Solution 2: Search services and product page services provide external operation interfaces, and the background calls the interface after adding, deleting, and modifying products

The above two methods have the same serious problem: code coupling, search and product page services need to be embedded in back-end services, which violates the 独立principle of microservices.

So, we will solve this problem in another way: message queue

1.2. Message Queue (MQ)

1.2.1. What is a message queue

Message queue, namely MQ, Message Queue.

The message queue is typical: producer, consumer model. Producers continue to produce messages to the message queue, and consumers continue to obtain messages from the queue. Because the production and consumption of messages are asynchronous, and they only care about the sending and receiving of messages, there is no intrusion of business logic, so that the decoupling of producers and consumers is realized.

Combining the aforementioned issues:

• After adding, deleting and modifying the products, there is no need to operate the index library or static pages, just send a message, and don't care who receives the message.
• The search service and the static page service receive the message and process the index library and the static page respectively.

If there are other systems that also rely on the data of the commodity service in the future, they can also monitor the message, and the commodity service does not require any code modification.

1.2.2.AMQP and JMS

MQ is a model of message communication, not a specific implementation. There are two mainstream ways to implement MQ: AMQP and JMS.

The difference and connection between the two:

• JMS defines a unified interface to unify the message operation; AMQP is to unify the format of data interaction through the specified protocol
• JMS restricts the use of the Java language; AMQP is only a protocol and does not specify the implementation method, so it is cross-language.
• JMS provides two message models; the AMQP message model is more abundant

1.2.3. Common MQ products

• ActiveMQ: based on JMS
• RabbitMQ: Based on AMQP protocol, erlang language development, good stability
• RocketMQ: Based on JMS, Alibaba products, currently handed over to the Apache Foundation
• Kafka: Distributed messaging system, high throughput

1.2.4.RabbitMQ

RabbitMQ is a message management system based on AMQP

Official website: http://www.rabbitmq.com/

Official tutorial: http://www.rabbitmq.com/getstarted.html

1.3. Download and install

Refer to another blog: rabbitMQ installation
https://blog.csdn.net/qq_38454176/article/details/105338529

2. Five message models

RabbitMQ provides six message models, but the sixth one is actually RPC, not MQ, so I won't learn it. Then there are only 5 types left.

For specific learning and use of RabbitMQ, please see another blog:
day72 JavaWeb framework stage-RabbitMQ message queue
https://blog.csdn.net/qq_38454176/article/details/105339764

3. Project transformation

Next, we will transform the project to achieve data synchronization of search services and product static pages.

3.1. Thinking analysis

Sender: Commodity microservices

• When will it be sent?

  When the commodity service writes the commodity: add, delete, or modify, a message needs to be sent to notify other services.

• What to send?

  Other services may require new product data when adding, deleting, or modifying products, but if the content of the message contains all product information, the amount of data is too large, and not every service requires all the information. Therefore, we only send the product id , and other services can query the information they need based on the id.

Recipient: search microservices, static page microservices

What to do after receiving the message?

• Search microservices:
  • Add/Change: Add new data to the index database
  • Delete: delete index database data
• Static page microservice:
  • Add/Change: Create a new static page
  • Delete: delete the original static page

3.2. Sending messages for goods and services

We first leyou-item-serviceimplement sending messages in commodity microservices .

3.2.1. Introduce dependencies

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-amqp</artifactId>
</dependency>

3.2.2. Configuration file

We add some configuration about RabbitMQ in application.yml:

spring:
  rabbitmq:
    host: 192.168.56.101
    username: leyou
    password: leyou
    virtual-host: /leyou
    template:
      exchange: leyou.item.exchange
    publisher-confirms: true

• template: related AmqpTemplateconfiguration
  • exchange: The default switch name. After configuration here, this will be used for sending messages if no switch is specified
• publisher-confirms: Producer confirmation mechanism to ensure that the message will be sent correctly, if the sending fails, there will be an error receipt, which will trigger a retry

3.2.3. Transform GoodsService

Encapsulate a method to send a message to mq in GoodsService : (Need to inject the AmqpTemplate template)

private void sendMessage(Long id, String type){
    
    
    // 发送消息
    try {
    
    
        this.amqpTemplate.convertAndSend("item." + type, id);
    } catch (Exception e) {
    
    
        logger.error("{}商品消息发送异常，商品id：{}", type, id, e);
    }
}

No switch is specified here, so it is sent to the configuration by default:leyou.item.exchange

Note: All exceptions must be tried here, so that the sending of the message cannot affect the normal business logic

Then call when adding:

Called when modifying:

3.3. Search service receiving messages

What to do after the search service receives the message:

• Increase: add new data to the index library
• Delete: delete index database data
• Change: modify index database data

Because the new and modified methods of the index library are combined, we can process these two types of messages together, and delete them separately.

3.3.1. Introduce dependencies

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-amqp</artifactId>
</dependency>

3.3.2. Add configuration

spring:
  rabbitmq:
    host: 192.168.56.101
    username: leyou
    password: leyou
    virtual-host: /leyou

Here, the message is only received but not sent, so there is no need to configure template related content.

3.3.3. Writing a listener

Code:

@Component
public class GoodsListener {
    
    

    @Autowired
    private SearchService searchService;

    /**
     * 处理insert和update的消息
     *
     * @param id
     * @throws Exception
     */
    @RabbitListener(bindings = @QueueBinding(
            value = @Queue(value = "leyou.create.index.queue", durable = "true"),
            exchange = @Exchange(
                    value = "leyou.item.exchange",
                    ignoreDeclarationExceptions = "true",
                    type = ExchangeTypes.TOPIC),
            key = {
    
    "item.insert", "item.update"}))
    public void listenCreate(Long id) throws Exception {
    
    
        if (id == null) {
    
    
            return;
        }
        // 创建或更新索引
        this.searchService.createIndex(id);
    }

    /**
     * 处理delete的消息
     *
     * @param id
     */
    @RabbitListener(bindings = @QueueBinding(
            value = @Queue(value = "leyou.delete.index.queue", durable = "true"),
            exchange = @Exchange(
                    value = "leyou.item.exchange",
                    ignoreDeclarationExceptions = "true",
                    type = ExchangeTypes.TOPIC),
            key = "item.delete"))
    public void listenDelete(Long id) {
    
    
        if (id == null) {
    
    
            return;
        }
        // 删除索引
        this.searchService.deleteIndex(id);
    }
}

3.3.4. Writing create and delete index methods

Here because we want to create and delete indexes, we need to expand two methods in SearchService to create and delete indexes:

public void createIndex(Long id) throws IOException {
    
    

    Spu spu = this.goodsClient.querySpuById(id);
    // 构建商品
    Goods goods = this.buildGoods(spu);

    // 保存数据到索引库
    this.goodsRepository.save(goods);
}

public void deleteIndex(Long id) {
    
    
    this.goodsRepository.deleteById(id);
}

The method of creating an index can be copied and modified from the test class that imported the data before.

3.4. Static page service receives messages

Processing after receiving the message by the commodity static page service:

• Increase: create a new static page
• Delete: delete the original static page
• Change: create a new static page and overwrite the original

However, the method we wrote to create a static page also has the function of overwriting the previous page. Therefore, the added and modified messages can be processed in one method, and the deleted messages can be processed in another method.

3.4.1. Introduce dependencies

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-amqp</artifactId>
</dependency>

3.4.2. Add configuration

spring:
  rabbitmq:
    host: 192.168.56.101
    username: leyou
    password: leyou
    virtual-host: /leyou

Here, the message is only received but not sent, so there is no need to configure template related content.

3.4.3. Writing a listener

Code:

@Component
public class GoodsListener {
    
    

    @Autowired
    private GoodsHtmlService goodsHtmlService;

    @RabbitListener(bindings = @QueueBinding(
            value = @Queue(value = "leyou.create.web.queue", durable = "true"),
            exchange = @Exchange(
                    value = "leyou.item.exchange",
                    ignoreDeclarationExceptions = "true",
                    type = ExchangeTypes.TOPIC),
            key = {
    
    "item.insert", "item.update"}))
    public void listenCreate(Long id) throws Exception {
    
    
        if (id == null) {
    
    
            return;
        }
        // 创建页面
        goodsHtmlService.createHtml(id);
    }

    @RabbitListener(bindings = @QueueBinding(
            value = @Queue(value = "leyou.delete.web.queue", durable = "true"),
            exchange = @Exchange(
                    value = "leyou.item.exchange",
                    ignoreDeclarationExceptions = "true",
                    type = ExchangeTypes.TOPIC),
            key = "item.delete"))
    public void listenDelete(Long id) {
    
    
        if (id == null) {
    
    
            return;
        }
        // 删除页面
        goodsHtmlService.deleteHtml(id);
    }
}

3.4.4. Add and delete page method

public void deleteHtml(Long id) {
    
    
    File file = new File("C:\\project\\nginx-1.14.0\\html\\item\\", id + ".html");
    file.deleteOnExit();
}

3.5. Testing

3.5.1. View RabbitMQ console

Restart the project and log in to the RabbitMQ management interface: http://192.168.56.101:15672

As you can see, the switch has been created:

The queue has also been created:

And the queues have been bound to the switch:

3.5.2. Modify the data and try

Modify the price of the product data in the background, and check whether it is unified on the search and product details pages.