Redis project actual combat - Coupon flash sale

Redis auto-increment function solves global unique ID

• If you use MySQL's self-increasing ID, the regularity of the ID is too obvious.Some information will be leaked(Such as sales volume, etc.)
• When the amount of data is too large, it cannot be stored in one table and multiple tables are needed. The self-growth of multiple MySQL tables is independent.Duplicate IDs will appear
• We need a tool that can generate a globally unique ID in a distributed system. It must be unique and incremental.
• In a certain project, no matter how many tables there are in the database, there is only one Redis, so the ID generated by the Redis incremental function must be globally unique.
• In order to ensure that the increment is simultaneous and irregular, and to ensure security, you canSplicing some other information based on the Redis auto-increment value
  

The main idea of ​​Redis to implement coupon flash sale

Problems that arise during implementation and their solutions

oversold problem

• In high-concurrency scenarios, multiple threads operate shared resources (inventory) at the same time, causing the actual quantity sold to exceed the inventory quantity.

Option 1 pessimistic lock

• The attitude is relatively pessimistic, thinking that thread safety problems will definitely occur, and acquiring locks in advance before operating data
• Example: Synchronized, Lock
• advantage: High security
• shortcoming: Low performance, simple implementation

Option 2 Optimistic locking

• The attitude is relatively optimistic, and I believe that thread safety problems may not occur, so I do not lock it and only lock it when the data is updated.judgmentDid any other thread modify the data before it? If it has not been modified, it is considered safe. Update the data directly. If it has been modified, it is unsafe. Retry or report an exception.
• version number method: Add a version field to the inventory. Thread 1 queries and records the inventory and version number, and then sets the inventory -1 and version number +1 to indicate that thread 1 has modified the data once, and then determines the version number before updating the data. Is it the version number + 1 recorded at that time? If so, it means that no concurrent thread has modified the data during the period. It is safe and you can update with confidence. If not, it means that there happened to be a concurrent thread that has modified the data during the period. It is not safe. Try again or Report exception
• CAS Act: A simplified version of the version number method, removing the redundant field of version number, directly replacing the version number with the inventory itself, and determining whether to update based on whether the inventory itself has changed.
• advantage: High performance
• shortcoming: Implementation is complicated

One question per person

• Common business problem: one user can only place an order for the same coupon
• After the inventory adequacy judgment is successful, another judgment is added, and the user ID and coupon ID are jointly queried to determine whether the user has purchased a coupon.
• In stand-alone mode, you can add Synchronized lock to ensure thread safety
• In cluster mode, Synchronized locks are invalid, and distributed locks need to be used to ensure thread safety. The reason why Synchronized locks are invalid is because each server has its own constant pool, and the lock monitor is stored in the constant pool. When users try to obtain the lock, they access the lock monitor. Therefore, the main problem is because of the lock monitors of multiple servers. is independent, so users on multiple servers can acquire locks at the same time, causing thread safety issues.

Distributed lock

• In a stand-alone situation, there is only one JVM, there is only one lock monitor in the JVM, and only one program can acquire the lock. But in the case of a cluster, there are multiple JVMs and multiple lock monitors in multiple JVMs. The program can acquire multiple locks, and even the same program can acquire multiple locks, and thread safety issues will arise.
• Need to do one outside of multiple JVMsShared, multi-process visible, mutually exclusive lock monitor - distributed lock
• There are three major ways to implement distributed locks: MySQL, Redis, and Zookeeper. MySQL and Zookeeper are more secure than Redis, and Redis has better performance than both.
  

How to implement distributed locks with Redis?

• Get mutex lock: SET lock thread1 NX EX 10, NX is mutually exclusive, ensuring that only one thread can obtain the lock, EX is to set the timeout.
• release lock: Directly delete manually.
• deadlock problem: If the thread goes down after acquiring the lock, deadlock is likely to occur. The expiration time should be increased and the lock will be automatically released after timeout.
• Accidental deletion problem: If thread 1 acquires the lock, but the business execution time is too long and exceeds the TTL, the lock will be automatically released. At this time, thread 2 attempts to acquire the lock successfully and executes the business normally. However, during the period, thread 1 completes the business execution and releases the lock normally. operation, at this time the lock of thread 2 will be deleted by mistake. In order to avoid this situation, an identification should be added when acquiring the lock to indicate who owns the lock, and only it is qualified to release the lock. Therefore, a judgment step needs to be added before releasing the lock.
• Problems with distributed locks implemented based on setnx: No reentrancy (the same thread cannot acquire the same lock multiple times), no retry (only try once to acquire the lock, no retry if failed), timeout release (long business execution time will cause the lock to be released, there is security hidden danger)
• Redission component: A collection of distributed tools implemented based on Redis

Redis optimization flash sale

• The main idea of ​​optimization: use time-consuming operations such as inventory reduction and order creation that involve the database.asynchronousIndependent threads do it slowly. Redis only needs to determine whether the user has successfully grabbed it and return the result.
• The original flash sale process: mainly a series of operations in Tomcat, four of which directly operate the database, which takes a very long time. It is equivalent to a restaurant. When a customer comes, a waiter is assigned to provide one-stop service for the customer. From ordering (checking flash sale qualifications) to cooking (reducing inventory and creating orders), this waiter is very efficient. low.
• Optimized flash sale process: Add Redis to NGINX and Tomcat Home to determine whether the user can grab the coupon, and save the judgment result together with the coupon id, user id, and order id to the blocking queue, and then Tomcat will Read messages from the queue and perform time-consuming operations of reducing inventory and creating orders.
  
• The operation of Redis to determine flash sale inventory can be encapsulated into a Lua script for execution to ensure the atomicity of the operation.
  
• What are the problems with asynchronous flash sales based on blocking queues?
• When the blocking queue is used in JDK, it will occupy JVM memory, and a large number of messages will cause memory overflow.

Message queue implements asynchronous flash killing

• Message queue: stores and manages messages
• Producer: sends messages to the message queue
• Consumer: Gets messages from the message queue and processes them
• Redis implements message queues in three ways: List, publish-subscribe mode, and Stream

List

• The chained double-ended queue uses LPUSH to store and RPOP to retrieve, but it has no blocking effect (it will not block waiting when the queue is empty), and BRPOP has a blocking effect.
• advantage: It exists independently of the JVM, does not occupy JVM memory, does not worry about the upper limit, can be persisted, and can also ensure the orderliness of messages.
• shortcoming: Unable to avoid message loss, only supports one-to-one

publish-subscribe model

• Consumers subscribe to one or more channels, and producers send messages to the corresponding channels.
• advantage: Supports one-to-many, one producer can send messages to multiple consumers. Naturally supports blocking
• shortcoming: Data persistence is not supported, message loss cannot be avoided, and there is an upper limit for message accumulation.

Stream

• advantage: Messages can be traced back, support one-to-many, support blocking reading
• shortcoming: Messages may be missed
• consumer group: Divide multiple consumers into a group and listen to the same message queue. Then multiple consumers will compete for these messages, which can speed up message processing and avoid message accumulation. The consumer group will also maintain an identifier to record the last processed message, which can quickly recover from emergencies and avoid missing messages. In addition, after the consumer gets the message, Redis will not directly ignore the message, but will set the message to pending status, indicating that the message has been fetched but has not been processed. After processing, Redis will confirm the message with XACK and mark it. Once the message has been processed, Redis will safely remove the message from the queue to prevent message loss.
• Consumer Group Advantages: Messages can be traced back, allowing multiple consumers to compete for messages, speeding up consumption, blocking reading, and not missing messages. There is a message confirmation mechanism to ensure that messages are consumed at least once.

Comparison summary of three message queues