"Redis transaction failure problem record"
In a time-limited snap-up scenario, if the write operation of hot data is performed in an RDBMS, multiple threads will compete with each other for InnoDB row locks. The higher the concurrency, the more threads waiting, which will lead to an increase in RT and a decrease in TPS. Eventually cause a system avalanche. Therefore, it is a good choice to place the inventory deduction action in Redis and use the optimistic locking method for deduction. After all, the throughput of Redis is there, and there is no row lock problem.
During this period of time, I was carrying out secondary optimization of inventory deduction (commissioning the success rate of inventory deduction, reducing the probability of watch collision at the same time), and found a problem. I used watch+multi directly in redis-cli to operate, and oversold actually occurred. , and produced a lot of strange phenomena:
1. After multi, it is submitted directly without joining the transaction queue;
2. Successfully joined the transaction queue, but did not achieve consistency;
3、...
This is a bit big. Quickly review the online inventory deduction logic code and check whether the spike order is oversold, but they are all normal, and using jedis to directly connect to redis for oversold testing, the above problem has not been reproduced. Kind of weird. After reviewing for a while, the problem was later located on the timeout of redis .
The transaction queue of Redis is stored on the client side. Simply put, after the timeout, the session is disconnected and the connection is rebuilt, and the transaction queue of the previous session is naturally destroyed, so this strange situation occurs , and the transaction feels invalid. In addition, the performance of each redis is different, because the server timeouts of different services are different, causing smoke bombs. The timeout on the server side is the "culprit". The reason why the jedis pressure test is not oversold is because the operation pause time will not be longer than the timeout time of the server, and it will not be considered by redis that the idle connection is released.
Of course, if the timeout of redis is 0 and never times out, it means that the client transaction queue will never expire. Of course, this is unreasonable.