What is idempotence
Idempotence (idempotent, idempotence) is a mathematical and computer science concept commonly found in abstract algebra.
The characteristic of an idempotent operation in programming is that the impact of any multiple executions is the same as the impact of one execution . An idempotent function, or idempotent method, is a function that can be executed repeatedly with the same parameters and obtain the same result. These functions will not affect the system state, and there is no need to worry about changes to the system caused by repeated execution.
Reasons for idempotence
- There are no restrictions on the front end, causing users to submit repeatedly
- Use the browser to go back, or press F5 to refresh, or use history to submit the form repeatedly.
- Network fluctuations, causing repeated requests
- Retry after timeout, causing repeated calls to the interface
- The setting of scheduled tasks is unreasonable, resulting in repeated data processing.
- When using message queue, messages are consumed repeatedly
How to ensure idempotence
1. Front-end processing
- Click the submit button to gray it out, or add loading
- Page redirection (PRG), PRG mode means that
POST-REDIRECT-GETwhen the user submits the form, it will重定向到另外一个提交成功页面，而不是停留在原先的表单页面。这样就避免了用户刷新导致重复提交。同时防止了通过浏览器按钮前进/后退导致表单重复提交。
2. Select first and then insert + unique index conflict
Before saving the data, we need to first select whether the data exists. If the data already exists, failure is returned (the specific operation depends on the business situation). If the data does not exist, the insert operation is performed.
However, in high-concurrency scenarios, there may be two requests for select, but no data is found, and then both perform insert operations , so duplicate data will be generated at this time, so in the database, we need to add a unique index to ensure idempotence.
The flow chart is as follows:
This solution is suitable for new operation interfaces.
3. Create a duplicate table
In some business scenarios, duplicate data is allowed to exist. Duplicate data is not allowed only in a certain link of the process. In this case, it is inappropriate to directly add a unique index to the table, so a deduplication table needs to be created. .
CREATE TABLE `order_heavy` (
`id` bigint(15) NOT NULL AUTO_INCREMENT COMMENT '自增主键',
`order_id` bigint(15) NOT NULL COMMENT '订单号',
`create_time` datetime DEFAULT NULL COMMENT '创建时间',
PRIMARY KEY (`id`),
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='订单去重表';The flow chart is as follows:
Special attention should be paid to that the anti-duplication table and the business table must be in the same database, and the operations must be in the same transaction.
This solution is suitable for insertion scenarios where there is a unique identifier in the business. For example, in the payment business, if an order will only be paid once, the order ID can be used as the unique identifier.
4. Use pessimistic locking
Pessimistic lock, as its name suggests, has strong exclusive and exclusive characteristics. It refers to a conservative attitude towards data being modified by the outside world (including other current transactions of the system and transaction processing from external systems). Therefore, the data is kept in a locked state during the entire data processing process. The implementation of pessimistic locking often relies on the locking mechanism provided by the database (only the locking mechanism provided by the database layer can truly guarantee the exclusivity of data access. Otherwise, even if the locking mechanism is implemented in this system, there is no guarantee that the external system will not modify it. data).
In the transaction scenario, the user's account balance is 100 yuan, and 50 yuan is transferred out. Under normal circumstances, the user's balance remains 50 yuan.
update t_account set amount-50 where id = 123;If there are multiple identical requests at this time, it may cause the user's amount to become negative. Therefore, pessimistic locking can be used at this time to lock the user's row data. Only one request is allowed to obtain the lock at the same time, while other requests wait.
select * from t_account where id = 123 for update;The flow chart is as follows:
Special attention should be paid to: If you are using a mysql database, the storage engine must use innodb, because it only supports transactions. In addition, the id field here must be the primary key or unique index, otherwise the entire table will be locked.
Because pessimistic locking requires locking a row of data in the same transaction, if the transaction is relatively long, it will cause a large number of requests to wait, affecting interface performance.
5. Use optimistic locking
Optimistic Locking Compared with pessimistic locking, the optimistic locking mechanism adopts a more relaxed locking mechanism. In most cases, pessimistic locking relies on the locking mechanism of the database to ensure maximum exclusivity of operations. But what follows is a large overhead in database performance, especially for long transactions, which is often unbearable. The optimistic locking mechanism solves this problem to a certain extent. Optimistic locking is mostly implemented based on the data version (Version) recording mechanism. What is a data version? That is to add a version identifier to the data. In a version solution based on a database table, this is usually achieved by adding a "version" field to the database table. When the data is read out, this version number is also read out, and when it is updated later, this version number is incremented by one. At this time, the version data of the submitted data is compared with the current version information of the corresponding record in the database table. If the version number of the submitted data is equal to the current version number of the database table, it will be updated, otherwise it will be considered as expired data.
Optimistic locking mainly operates based on version identification (version), that is, every time you query data, you must first query the version identification (version), and then perform an update operation based on the version identification (version).
select id,amount,version from t_account id = 123;
update t_account set amount=amount-50,version=version+1 where id=123 and version = 1;When multiple identical requests query information, the version identifiers are the same. When one of the requests completes the update operation, the number of items affected by subsequent requests is 0.
The flow chart is as follows:
6. According to the status mechanism
Many times, business processes have state flows. In this case, state machines can be used to ensure idempotence.
For example, in the order business, there are statuses "1-order placed, 2-paid, 3-completed, 4-cancelled". According to the business process, the status flows sequentially, so during the update operation, we must This time's status is used to update the next status.
update t_order set status = 3 where id = 123 and status = 2;The flow chart is as follows:
7. Use distributed locks
The logic of the distributed lock is that each request attempts to obtain the lock through the unique business ID. If the acquisition is successful, subsequent business logic operations will be performed. If the acquisition fails, the request will be discarded and returned directly.
Distributed locks are usually implemented based on redis.
The flow chart is as follows:
Distributed locks are controlled by setting the expiration time of redis. If the expiration time is set too short, repeated requests cannot be effectively prevented; if the expiration time is set too long, it will affect the redis storage space and even affect subsequent business operations. Therefore, a reasonable expiration time needs to be set based on specific business conditions.
8. Based on token mechanism
This scenario contains two request phases:
1. The client requests the server to apply for a token
2. The client carries the token and requests again, and the server verifies the token and then performs the operation.
The flow chart is as follows:
To verify whether the token exists on the server side, you need to use the method of deleting the key, that is, redis.del(key). If the deletion is successful, it means that the verification token has passed; you cannot
use the operation of checking first and then deleting, that is, first redis.get(key), and then redis.del(key), this method cannot guarantee idempotence under high concurrency.