In a distributed system, there are a large number of external requests from machine clusters to the system, and how to ensure the consistency of data processed by multiple machines is a key issue that needs to be considered when load balancing to each machine.
Case: The external system synchronizes peripheral data to the system by means of message queues. It may send multiple pieces of data at one time, and distribute them to multiple machines in the system for processing.
Scenario: A product corresponds to multiple pieces of supplier data. After being stored in the database, the key value inserted into the redis cache cache is the product, and the value value is the supplier's JsonList string. Two pieces of data come from the periphery and are allocated to two machines for processing. Each machine is processed according to the
Open database transaction--->Insert data--->Compensate cache--->Transaction commit logic processing
In the case of concurrency, it may occur that the transaction enters the database at the same time and inserts data. Due to the isolation type of the transaction, it is impossible to see the data inserted by another transaction if it is not committed. Therefore, each query is inserted into the cache and the supplier has only one record in the final cache.
Database simulation (mysql+dbvisual)
1. Create an experiment table
create table cmmdty_supplier (
id int auto_increment primary key,
cmmdty_code varchar(100),--commodity code
supplier_code varchar(100) -- supplier code
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
2. Open a window of dbvisual and insert the A supplier data of commodity 0001 into the table
set autocommit=0; --manually control transaction commit
insert into cmmdty_supplier(cmmdty_code,supplier_code)
values ('0001','A');
Open another dbvisual window and insert the B supplier data of commodity 0001 into the table
set autocommit=0;
insert into cmmdty_supplier(cmmdty_code,supplier_code)
values ('0001','B');
The data in the query table of each window can only see the data inserted by each, and can only be seen after each commit.
Solution 1:
Use select for update to lock the data to be processed. After testing, it is found that only existing data can be locked, that is, table data can be updated or deleted, and new data cannot be locked.
Solution 2:
Since redis is based on a single thread and the setnx command realizes that the setting can be successful only if the value does not exist, the setnx function of redis is used to realize the function of distributed lock, and the specific implementation process
