what is idempotency
Idempotency is applied in software systems. I simply define it as: a function or an interface is called once or infinite times with the same parameters, and the consequences are the same . In practical applications, idempotency is generally performed on the interface. Isometric design. For example, in the system, when caller A calls the interface of system B to perform the user's deduction operation, due to the instability of the network, A retries the request N times, so no matter how many times B receives the request, it should be It is guaranteed that the user will only be charged once.
Idempotent Design
Idempotency is generally used in protocol design. Does the TCP protocol support idempotency? The answer is yes. When the network is unstable, the operating system can re-send TCP message fragments recklessly. The core of the TCP protocol's ability to ensure idempotency lies in the sequence number field, and a sequence number will not be repeated for a long period of time. For the protocol design of the application layer, the principle is similar to that of TCP, and we need a sequence number that does not repeat. To put it simply, in the processing of a business process, we need a non-repetitive business serial number to ensure idempotency.
Take a practical application scenario: User A initiates a game recharge request on the web page, the browser guides the user to go to the bank to pay, and the system recharges the user after the payment is successful.
In terms of protocol design, we run through the entire business process through a globally unique recharge order number, so that the business supports idempotence.
In terms of application implementation, we list the steps to call back the system and recharge after successful bank payment.
- func pay_notify(orderid, value, state){//Problematic implementation
- order = db.query("select * from payorder where orderid=$orderid");
- check(order,orderid,value,state);//Determine whether the payment amount is consistent with the order amount, and determine whether it is a successful payment callback.
- if(order.state=='Unpaid'){
- db.update("update payorder set state='已支付' where orderid=$orderid");
- charge(order.username,value);//Execute recharge
- }else{
- return result("Order has been processed")//Return the order has been processed, or return the processing is successful
- }
- }
The problem with the above implementation is that when the callback is concurrent, order.state is already dirty, and it is possible to recharge repeatedly. This implementation cannot guarantee idempotency 100%.
List three ways to improve:
1. Pessimistic lock, select for update, lock the record corresponding to the order during the entire execution process.
2. Optimistic lock, affectrows = db.update("update payorder set state='paid' where orderid=$orderid and state='unpaid' "), if affectrows=1, execute recharge, otherwise return processed.
3. Define the notifylog table. The orderid is unique key or primary key. Before executing, insert first. If the insert is successful, execute the recharge, otherwise return processed.
The above simple examples are used to illustrate common application implementations of idempotency. In SOA systems, many atomic functions may be split into different processes. For example, the function of charge recharge may be in another process. The link will be longer, the callback may be successful, but the recharge fails. Similarly, as long as the recharge interface guarantees idempotency, for a request that has been called back but the recharge result has not been returned, the receiving program should be called back and the recharge request should be repeated. More in-depth and more complex application scenarios will be discussed in detail in Data Consistency.
Summarize
When designing the protocol at the business layer, the requester is required to define a non-repetitive business serial number. When the application is implemented, use the database optimistic lock, insert the log of the unique key, etc. to ensure the idempotency during concurrency.
幂等性把关环节，在协议设计评审中，评审重要业务RPC或者http接口是否支持幂等，代码评审中，重点把关请求并发时，是否仍旧能够保证幂等性。设计人员和具体实现人员在实现过程中，也应该时刻自审幂等性的实现是否过关。
-----------------------------------------------------
补充：
为啥要做幂等呢？
原因很简单，在系统调用没有达到期望的结果后，会重试。那重试就会面临问题，重试之后不能给业务逻辑带来影响，例如创建订单，第一次调用超时了，但是调用的系统不知道超时了是成功了还是失败了，然后他就重试，但是实际上第一次调用订单创建是成功了的，这时候重试了，显然不能再创建订单了。
查询
查询的API，可以说是天然的幂等性，因为你查询一次和查询两次，对于系统来讲，没有任何数据的变更，所以，查询一次和查询多次一样的。
删除数据
删除数据，仅仅第一次删除是真正的操作数据，第二次甚至第三次删除，直接返回成功，这样保证了幂等。
MVCC方案
多版本并发控制（Multiversion concurrency control，MVCC）。人们一般把基于锁的并发控制机称成为悲观机制，而把MVCC等机制称为乐观机制。这是因为锁机制是一种预防性的，读会阻塞写，写也会阻塞读，当锁定粒度较大，时间较长是并发性能就不会太好；而MVCC是一种后验性的，读不阻塞写，写也不阻塞读，等到提交的时候才检验是否有冲突，由于没有锁，所以读写不会相互阻塞，从而大大提升了并发性能。
MVCC的一种简单实现是基于CAS（Compare-and-swap）思想的有条件更新（Conditional Update）。这也是在系统设计的时候，合理的选择乐观锁，通过version或者其他条件，来做乐观锁，这样保证更新即使在并发的情况下，也不会有太大的问题。例如update table_xxx set
name=#name#,version=version+1 where version=#version# ,或者是 update table_xxx set quality=quality-#subQuality# where quality-#subQuality# >= 0 。
单独的去重表
如果涉及到的去重的地方特别多，例如ERP系统中有各种各样的业务单据，每一种业务单据都需要去重，这时候，可以单独搞一张去重表，在插入数据的时候，插入去重表，利用数据库的唯一索引特性，保证唯一的逻辑。
插入数据的唯一索引
插入数据的唯一性，可以通过业务主键来进行约束，例如一个特定的业务场景，三个字段肯定确定唯一性，那么，可以在数据库表添加唯一索引来进行标示。
API层面的幂等
这里有一个场景，API层面的幂等，例如提交数据，如何控制重复提交，这里可以在提交数据的form表单或者客户端软件，增加一个唯一标示，然后服务端，根据这个UUID来进行去重，这样就能比较好的做到API层面的唯一标示。
状态机幂等
在设计单据相关的业务，或者是任务相关的业务，肯定会涉及到状态机，就是业务单据上面有个状态，状态在不同的情况下会发生变更，一般情况下存在有限状态机，这时候，如果状态机已经处于下一个状态，这时候来了一个上一个状态的变更，理论上是不能够变更的，这样的话，保证了有限状态机的幂等。
分布式锁
Take the example of inserting data. If it is a distributed system, it is difficult to build a unique index. For example, the unique field cannot be determined. At this time, a distributed lock can be introduced to insert or update data in the business system through a third-party system. , acquire distributed locks, then perform operations, and then release the locks. This is actually the idea of introducing multi-threaded concurrent locks into multiple systems, that is, a solution to the distributed system.
