Article Directory
- I. Introduction
- 2. Problems encountered in search and commodity services
- Three, message queue processing data consistency
- Four, interview golden finger
- V. Summary
I. Introduction
2. Problems encountered in search and commodity services
At present, we have completed the development of product details and search system. Let's think about it, is there a problem?
Question 1 (data consistency between the database and the ES index library): The original data of the product is stored in the database, and all additions, deletions, changes and checks are completed in the database. The source of the search service data is the index database. If the database commodities change, the index database data cannot be updated in time.
Question 2 (data consistency between the database and the Nginx static page): The product details have been staticized on the page, and the static page data will not change with the database product.
If we modify the price of the product in the background, the ES search page and Nginx product detail page still display the old price, which is obviously wrong.
Two solutions:
Solution 1 (service layer, modify immediately, modify when writing data operation): Whenever the background performs additions, deletions, and modifications to the product data, and the data in the database is written, the index library data and the static page
solution should be modified at the same time 2 (controller layer, modify immediately, modify when writing data operation): ES search service and Nginx product detail page service provide external operation interfaces. After the product is added, deleted and modified in the background, the two interface
summary is called . Both are the same. There is also a delayed modification, which is modified when the data is read again.
The above two methods have the same serious problem: code coupling, the need to embed search and product page services in product services, which violates the principle of independence of microservices.
Therefore, we will solve this problem in another way: message queue: message queue
: asynchronous, decoupling (relative to multithreading in the same Java program), current limit protection database
Three, message queue processing data consistency
Next, we will transform the project to achieve data synchronization of search services and product static pages.
3.1. Thinking analysis
Sender: When will the product microservice
be sent? When the commodity service writes the commodity: add, delete, or modify, a message needs to be sent to notify other services.
What to send? Other services may require new product data when adding, deleting, or modifying products, but if the message content contains all product information, the amount of data is too large, and not every service requires all the information. Therefore, we only send the product id , and other services can query the information they need based on the id.
Recipient: Search microservices, static page microservices
What to do after receiving the message?
Consumer 1: Search microservices:
add/modify: add new data to the index database
Delete: delete index database data
Consumer 2: static page microservices:
add: create a new static page
Delete: delete the original static page
Change: Create a new static page and delete the original
3.2. Practice: Producer: Send message for goods and services
Let's first leyou-item-serviceimplement sending messages in commodity microservices .
3.2.1. Introduce dependencies
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-amqp</artifactId>
</dependency>
3.2.2. Configuration file
We add some configuration about RabbitMQ in application.yml:
spring:
rabbitmq:
host: 192.168.56.101
username: leyou
password: leyou
virtual-host: /leyou
template:
exchange: leyou.item.exchange
publisher-confirms: true
- template: related
AmqpTemplateconfiguration- retry: Retry on failure
- enabled: retry after failed to enable
- initial-interval: the interval between the first retry
- max-interval: the longest retry interval, beyond this interval will not retry
- multiplier: the multiple of the next retry interval, here is 2 that the next retry interval is twice the last time
- exchange: The default switch name. After configuration here, this will be used for sending messages if no switch is specified
- retry: Retry on failure
- publisher-confirms: Producer confirmation mechanism to ensure that the message will be sent correctly, if the sending fails, there will be an error receipt, which will trigger a retry
3.2.3. Transform GoodsService
Encapsulate a method to send a message to mq in GoodsService:
private void sendMessage(Long id, String type){
// 发送消息
try {
this.amqpTemplate.convertAndSend("item." + type, id);
} catch (Exception e) {
logger.error("{}商品消息发送异常，商品id：{}", type, id, e);
}
}
No switch is specified here, so it is sent to the configuration by default:leyou.item.exchange
Note: All exceptions must be tried here, so that the sending of the message cannot affect the normal business logic
For commodity microservices, it is called when adding new records, and finally, the message is sent to the message queue rabbitmq:
For commodity microservices, when modifying, modify the record and finally send a message to the message queue rabbitmq
3.3. Practice: Consumer 1: Search service receives messages
What to do after the search service receives the message:
add: add new data to the index database
delete: delete the index database data
change: modify the index database data
because the new and modified methods of the index database are combined, so we You can process these two types of messages together, and delete additional processing.
3.3.1. Introduce dependencies
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-amqp</artifactId>
</dependency>
3.3.2. Add configuration
spring:
rabbitmq:
host: 192.168.56.101
username: leyou
password: leyou
virtual-host: /leyou
Here, the message is only received but not sent, so there is no need to configure template related content. good
3.3.3. Writing a listener
Code:
@Component
public class GoodsListener {
@Autowired
private SearchService searchService;
/**
* 处理insert和update的消息
*
* @param id
* @throws Exception
*/
@RabbitListener(bindings = @QueueBinding(
value = @Queue(value = "leyou.create.index.queue", durable = "true"),
exchange = @Exchange(
value = "leyou.item.exchange",
ignoreDeclarationExceptions = "true",
type = ExchangeTypes.TOPIC),
key = {
"item.insert", "item.update"}))
public void listenCreate(Long id) throws Exception {
if (id == null) {
return;
}
// 新增代码，监听到后，创建或更新ES索引库中的索引
this.searchService.createIndex(id);
}
/**
* 处理delete的消息
*
* @param id
*/
@RabbitListener(bindings = @QueueBinding(
value = @Queue(value = "leyou.delete.index.queue", durable = "true"),
exchange = @Exchange(
value = "leyou.item.exchange",
ignoreDeclarationExceptions = "true",
type = ExchangeTypes.TOPIC),
key = "item.delete"))
public void listenDelete(Long id) {
if (id == null) {
return;
}
// 新增代码，监听到后，删除ES索引库中的索引
this.searchService.deleteIndex(id);
}
}
3.3.4. Writing create and delete index methods
Here because we want to create and delete indexes, we need to expand two methods in SearchService to create and delete indexes:
public void createIndex(Long id) throws IOException {
Spu spu = this.goodsClient.querySpuById(id);
// 构建商品
Goods goods = this.buildGoods(spu);
// 保存数据到索引库
this.goodsRepository.save(goods);
}
public void deleteIndex(Long id) {
this.goodsRepository.deleteById(id);
}
The index creation method can be copied and modified from the test class that imported the data before.
3.4. Practice: Consumer 2: Static page service receives messages
The processing of the commodity static page service after receiving the message:
Add: create a new static page
Delete: delete the original static page
Change: create a new static page and delete the original
ps: The method we wrote to create a static page also has the function of overwriting the previous page. Therefore: the added and modified message can be processed in one method, and the deleted message can be processed in another method.
3.4.1. Introduce dependencies
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-amqp</artifactId>
</dependency>
3.4.2. Add configuration
spring:
rabbitmq:
host: 192.168.56.101
username: leyou
password: leyou
virtual-host: /leyou
ps: Like the ES index library, here only messages are received but not sent, so there is no need to configure template related content.
3.4.3. Writing a listener
Code:
@Component
public class GoodsListener {
@Autowired
private GoodsHtmlService goodsHtmlService;
@RabbitListener(bindings = @QueueBinding(
value = @Queue(value = "leyou.create.web.queue", durable = "true"),
exchange = @Exchange(
value = "leyou.item.exchange",
ignoreDeclarationExceptions = "true",
type = ExchangeTypes.TOPIC),
key = {
"item.insert", "item.update"}))
public void listenCreate(Long id) throws Exception {
if (id == null) {
return;
}
// 创建页面
goodsHtmlService.createHtml(id);
}
@RabbitListener(bindings = @QueueBinding(
value = @Queue(value = "leyou.delete.web.queue", durable = "true"),
exchange = @Exchange(
value = "leyou.item.exchange",
ignoreDeclarationExceptions = "true",
type = ExchangeTypes.TOPIC),
key = "item.delete"))
public void listenDelete(Long id) {
if (id == null) {
return;
}
// 删除静态页面
goodsHtmlService.deleteHtml(id);
}
}
3.4.4. Add a new method to delete pages
public void deleteHtml(Long id) {
File file = new File("C:\\project\\nginx-1.14.0\\html\\item\\", id + ".html");
file.deleteOnExit();
}
3.5. Test (check RabbitMQ console)
Restart the project and log in to the RabbitMQ management interface: http://192.168.56.101:15672
First, see that the switch has been created:
Second, see that the queue has also been created:
Third, see that the queues have been bound to the switch:
Fourth, try to modify the data, modify the price of the product data in the product microservice, and check whether it is unified in the search and product details pages.
Four, interview golden finger
4.1 Problems encountered in search and commodity services
At present, we have completed the development of product details and search system. Let's think about it, is there a problem?
Question 1 (data consistency between the database and the ES index library): The original data of the product is stored in the database, and all additions, deletions, changes and checks are completed in the database. The source of the search service data is the index database. If the database commodities change, the index database data cannot be updated in time.
Question 2 (data consistency between the database and the Nginx static page): The product details have been staticized on the page, and the static page data will not change with the database product.
If we modify the price of the product in the background, the ES search page and Nginx product detail page still display the old price, which is obviously wrong.
Two solutions:
Solution 1 (service layer, modify immediately, modify when writing data operation): Whenever the background performs additions, deletions, and modifications to the product data, and the data in the database is written, the index library data and the static page
solution should be modified at the same time 2 (controller layer, modify immediately, modify when writing data operation): ES search service and Nginx product detail page service provide external operation interfaces. After the product is added, deleted and modified in the background, the two interface
summary is called . Both are the same. There is also a delayed modification, which is modified when the data is read again.
The above two methods have the same serious problem: code coupling, the need to embed search and product page services in product services, which violates the principle of independence of microservices.
Therefore, we will solve this problem in another way: message queue: message queue
: asynchronous, decoupling (relative to multithreading in the same Java program), current limit protection database
4.2 Solution
Sender: When will the product microservice
be sent? When the commodity service writes the commodity: add, delete, or modify, a message needs to be sent to notify other services.
What to send? Other services may require new product data when adding, deleting, or modifying products, but if the message content contains all product information, the amount of data is too large, and not every service requires all the information. Therefore, we only send the product id , and other services can query the information they need based on the id.
Recipient: Search microservices, static page microservices
What to do after receiving the message?
Consumer 1: Search microservices:
add/modify: add new data to the index database
Delete: delete index database data
Consumer 2: static page microservices:
add: create a new static page
Delete: delete the original static page
Change: Delete the original page + create a new static page
4.3 Specific solutions
For commodity microservices, call when adding new records, and finally send a message to the message queue rabbitmq:
For commodity microservices , when modifying, modify the record and finally send a message to the message queue rabbitmq
What to do after the search service receives the message:
add: add new data to the index database
delete: delete the index database data
change: modify the index database data
because the new and modified methods of the index database are combined, so we You can process these two types of messages together, and delete additional processing.
Here, the message is only received but not sent, so there is no need to configure template related content. good
The processing of the commodity static page service after receiving the message:
add: create a new static page
delete: delete the original static page
change: create a new static page and delete the original
ps: the method we wrote to create a static page can also overwrite the previous The function of the page, therefore: add and change messages can be processed in one method, and deleted messages can be processed in another method.
ps: Like the ES index library, here only messages are received but not sent, so there is no need to configure template related content.
Restart the project and log in to the RabbitMQ management interface: http://192.168.56.101:15672
First, see that the switch has been created;
second, see that the queue has also been created;
third, see that the queues have been bound Set it to the exchange;
fourth, try to modify the data, modify the price of the product data in the product microservice, and check whether it is unified in the search and product details pages.
V. Summary
RabbitMQ e-commerce practice (database-ES index library-Nginx product details page), completed.
Code every day, make progress every day! ! !