I. Background
We actually have a lot of operating systems, it is to do no matter how many times should produce the same effect or return the same results.
E.g:
- Distal resubmit the selected data, should produce the background only corresponding to a result of the reaction of this data.
- We initiate a payment request, should only be deducted once the user account money, when faced with network retransmission or retransmission system bug, it should only be deducted once the money;
- To send a message, it should only be made once, the same message sent to the user, the user will cry;
- Create a business order, one can only create a service request, it will create more than a big problem.
And so many important cases, the logic required to support power and other features.
Second, the notion of idempotent
Idempotent (idempotent, idempotence) is a mathematics and computer science concepts, common in abstract algebra.
An idempotent operation is characterized by any of its influence are repeatedly produced with the same impact in the first execution of programming. Idempotent other functions, methods, or power means you can be repeatedly performed using the same parameters, and can achieve the same results function.
These functions do not affect the system status, do not worry Repeat will cause the system to change. For example, "getUsername () and setTrue ()" function is a function idempotent.
More complex operations such as power guarantee is achieved with a unique transaction number (serial number).
I understand it: so is a power operation, no matter how many times to perform, and the effect of the results returned are the same
Third, the technical program
1. query query query several times and once, in the case of the same data, the query result is the same. select natural idempotent operations
2. Delete operation deletion idempotent, delete, delete one or many times the data are deleted. (Note may not return results, deleting data does not exist, 0 is returned, a number of deleted data and return results more)
3. unique index to prevent new dirty data such as: Alipay capital account, Alipay is also user accounts, each user can have only one fund account, how to prevent the user to create multiple funds account, then to the capital account in the user table ID plus a unique index, so a user to add a successful capital account records
Highlights: a unique combination of a unique index or index to prevent the presence of additional data dirty data (when there is a unique index table, the new concurrent error, then check it once, the data should already exist, and you can return results)
4. token mechanism to prevent duplicate submission page
Operational requirements:
Click on the data page can only be submitted once
The causes: Due to repeated clicks or network retransmission, retransmission or nginx, etc. can cause data to be submitted to repeat
Solution: cluster environment: using token plus redis (redis single-threaded processing to wait in line) single JVM environment: the use of token or token plus redis plus jvm memory
Process flow:
- To apply for token before submitting data services, token or redis into jvm memory, token valid time
- After the submission of background check token, delete token, generate a new token return
token features:
To apply, once effective, can be limiting
Note: redis use the delete operation to determine the token, token deleted successfully represented by check, if select + delete to verify the token, concurrency problems, not recommended
The pessimistic locking the lock when acquiring data acquired
select * from table_xxx where id='xxx' for update;
Note: id field must be the primary key or unique index, or a lock table, will be dead
General use pessimistic locking when used along with transaction data lock time can be very long, selected according to the actual situation
6. optimistic optimistic lock only lock in the lock table update data that moment, other times not lock table, so relative to the pessimistic lock and more efficient.
Optimistic locking can achieve a variety of ways by version or other status conditions:
1, is achieved by version number
update table_xxx set name=#name#,version=version+1 where version=#version#
Below (from the Internet):
2, the conditions
update tablexxx set avaiamount=avaiamount-#subAmount# where avaiamount-#subAmount# >= 0
Requirements: quality- # subQuality #> =, do not fit this scenario version number, the update is only for data security check for inventory model, buckle share and rollback share, higher performance
Note: optimistic locking update operation, it is best to update a primary key or unique index, this is the line lock, lock the table will be updated when otherwise, the top two sql into the following two better
update tablexxx set name=#name#,version=version+1 where id=#id# and version=#version#
update tablexxx set avaiamount=avaiamount-#subAmount# where id=#id# and avai_amount-#subAmount# >= 0
7. Distributed Lock insert or take the example data, if the system is distributed, globally unique index is difficult to build, for example, a field can not determine uniqueness
This time distributed lock can be introduced, is inserted through a third party system (or Redis ZooKeeper) or update data in the data traffic system, distributed lock acquisition, and then do the operation, after the release of the lock
In fact, this idea is a lock to multi-threaded, the introduction of more multi-system, which is a distributed system must Solutions.
Important: a long flow processing requirements can not be performed concurrently, may be (user ID + suffixes) distributed lock acquired before the process performed according to a flag, acquires the lock will fail when the execution of other processes, i.e. the same time only the flow able to have a successful execution, execution is completed, release distributed lock (distributed lock to provide third-party systems)
8. select + insert is not complicated by the high background system, or some tasks JOB, in order to support idempotent, support repeatedly performed, a simple approach is to query the key data, judges whether has been performed, performing service processing, on it
Note: The core of high concurrent processes do not use this method
9. The power of the state machine and other related businesses in the design documents, or task-related business, will certainly involve state machine (state change diagram), it is to have a state business documents above, the status change will occur under different circumstances, there is a finite state machine in general
If the state machine is already in the next state, this time to a previous state of change, in theory, can not be changed, so, ensure that the power of finite state machines and so on.
Note: The class of business documents such as orders, there is a long state of circulation, must deeply understand the state machine, system design capabilities to improve business helps a lot
10. How to provide external interfaces api guarantee idempotent
Comes when access is required to submit payment requests merchant:: If UnionPay payment interfaces provide source sources, seq serial number
source + seq unique index to do in a database to prevent multiple payments, (concurrent, can only handle one request)
Provides an interface to support the emphasis on foreign powers such as call interface has two fields must pass, is the source of a source, a source side serial number is seq, the two fields in which to do joint system provider unique index
So that when a third-party call, first check the local system inside it, whether processed, the processing returns to the corresponding results; not dealt with, and dealt with accordingly, returns the result.
Note that in order idempotent friendship, be sure to inquire about whether treated this transaction, does not query directly into business systems, will complain, but actually has been processed.
to sum up
Idempotence should be a qualified programmer of a gene, in the design of the system, is the primary consideration, especially in a system like pay all the money involved in the treasure, banks, finance companies and other Internet, it is necessary to efficient, but also data accurate, so can not appear more than chargeback, and more play money and other issues, it will difficult to handle, the user experience is not good
I compiled a free Java Advanced information, covering Java, Redis, MongoDB, MySQL, Zookeeper, Spring Cloud, Dubbo distributed high concurrency and other tutorials, a total of 30G, needs its own collection.
Portal: https://mp.weixin.qq.com/s/osB-BOl6W-ZLTSttTkqMPQ