In the Pinxixi interview, the interviewer asked a series of classic questions: "How is the coupon inventory deducted? Have you ever understood the solution designed to solve the problem of over-issuing coupons? How did you test it?" Woolen cloth?"
At that time, I was quite confused when I heard these questions. Have you ever encountered the problem of over-delivery? How do I know the development and design plan? It’s quite naive to think about it now. In fact, now that I think about it, there are many similar problems in e-commerce, such as oversold products.
problem introduction
Take the issue of receiving coupons as an example,
Requirement description: A A total of 100 coupons are issued, and each user can receive up to 5 coupons.
When a user successfully receives the coupon, write the received record into another table (we call this table B for the time being).
In the process of receiving coupons, the general operation of the deduction process of coupon inventory is as follows:
1. Select to query the inventory of coupons.
2. Calculate whether the stock of coupons is sufficient. If the stock of coupons is insufficient, an exception of insufficient stock will be thrown. If the stock of coupons is sufficient, it will be judged whether it is time to collect the coupons and whether the quantity received by the user exceeds the maximum personal limit.
3. If 2 is true, subtract the deducted inventory to get the latest inventory remaining value.
4. Set sets the latest coupon inventory remaining value
The pseudo code is as follows:
The deduction coupon sql is as follows:
update coupon set stock = stock - 1 where id = #{coupon_id}
When the concurrency is relatively low, it is almost impossible to see that there is a problem, but when we enable multi-threading and request the interface for grabbing coupons, a problem arises. The coupon inventory with id 19 is negative. One more is sent, why?
In-depth interpretation of concurrency security issues
Why is there a problem of excessive coupon inventory when the concurrency is high? The reason is as follows screenshot:
The problematic link in the above figure is actually the step of judging the coupon inventory. Here comes the key point:
In the case of high concurrency, if two threads, thread A and thread B (which can be understood as two requests), come at the same time, for example, the request of thread A that came first passed the inspection. At this time, thread A has not deducted the inventory. After some operations, thread B also passed the method of checking whether the coupon can be claimed, and then thread A and thread B deducted the inventory in turn or at the same time. So there is a phenomenon that just appeared in the database, the coupon inventory is -1, just like the picture below.
How to solve the concurrency security problem?
Java code locking
synchronized (this){
LoginUser loginUser = LoginInterceptor.threadLocal.get();
CouponDO couponDO = couponMapper.selectOne(new QueryWrapper()
.eq("id", couponId)
.eq("category", categoryEnum.name()));
if(couponDO == null){
throw new BizException(BizCodeEnum.COUPON_NO_EXITS);
}
this.checkCoupon(couponDO,loginUser.getId());
//构建领券记录
CouponRecordDO couponRecordDO = new CouponRecordDO();
BeanUtils.copyProperties(couponDO,couponRecordDO);
couponRecordDO.setCreateTime(new Date());
couponRecordDO.setUseState(CouponStateEnum.NEW.name());
couponRecordDO.setUserId(loginUser.getId());
couponRecordDO.setUserName(loginUser.getName());
couponRecordDO.setCouponId(couponDO.getId());
couponRecordDO.setId(null);
int row = couponMapper.reduceStock(couponId);
if(row == 1){
couponRecordMapper.insert(couponRecordDO);
}else{
log.info("发送优惠券失败:{},用户:{}",couponDO,loginUser);
}
}Add a synchronized keyword, so that each request has to be queued to perform this deduction operation, which can solve the concurrency security problem to a certain extent, but because the synchronized keyword is locked based on the jvm level, when there are multiple jvm processes in a cluster environment, So this method is only suitable for stand-alone nodes.
SQL version number
update product set stock=stock-1 where stock=#{上一次的库存} and id = #{id} and stock>0This method has an ABA problem. We can add a version field, which will be incremented by 1 every time the data is modified, so that the ABA problem can be avoided. However, relying on the database for concurrent security guarantees will consume database resources and can be used within a certain amount of requests (subject to rigorous testing).
update product set stock=stock-1,versioin = version+1 where #{id} and stock>0 and version=#{上一次的版本号}Redis distributed lock
After the introduction of Redis, when receiving coupons, you will first go to Redis to obtain a lock, and only after the lock is obtained successfully can you operate on the database.
In distributed locks we should consider the following:
- Exclusiveness, in a distributed cluster, the same method can only be executed by one thread on a certain machine at the same time;
- Fault tolerance, when a thread is locked, if the machine suddenly crashes, if the lock is not released, the data will be locked at this time;
- Also pay attention to the granularity of the lock and the overhead of the lock;
- Meet high availability, high performance, reentrant.
The pseudo code is as follows:
@RestController
public class IndexController {
public static final String REDIS_LOCK = "coupon_lock";
@Autowired
StringRedisTemplate template;
@RequestMapping("/getCoupon")
public String getCoupon(){
// 每个人进来先要进行加锁，key值为"good_lock"
String value = UUID.randomUUID().toString().replace("-","");
try{
// 为key加一个过期时间
Boolean flag = template.opsForValue().setIfAbsent(REDIS_LOCK, value,10L,TimeUnit.SECONDS);
// 加锁失败
if(!flag){
return "抢锁失败！";
}
System.out.println( value+ " 抢锁成功");
String result = template.opsForValue().get("coupon:001");
int total = result == null ? 0 : Integer.parseInt(result);
if (total > 0) {
// 在此处需要处理抢购优惠券业务，处理时间较长。。。
int realTotal = total - 1;
template.opsForValue().set("coupon:001", String.valueOf(realTotal));
System.out.println("获取优惠券成功，库存还剩：" + realTotal + "件， 服务端口为8001");
return "获取优惠券成功，库存还剩：" + realTotal + "件， 服务端口为8001";
} else {
System.out.println("获取优惠券失败，服务端口为8001");
}
return "获取优惠券失败，服务端口为8001";
}finally {
// 谁加的锁，谁才能删除，使用Lua脚本，进行锁的删除
Jedis jedis = null;
try{
jedis = RedisUtils.getJedis();
String script = "if redis.call('get',KEYS[1]) == ARGV[1] " +
"then " +
"return redis.call('del',KEYS[1]) " +
"else " +
" return 0 " +
"end";
Object eval = jedis.eval(script, Collections.singletonList(REDIS_LOCK), Collections.singletonList(value));
if("1".equals(eval.toString())){
System.out.println("-----del redis lock ok....");
}else{
System.out.println("-----del redis lock error ....");
}
}catch (Exception e){
}finally {
if(null != jedis){
jedis.close();
}
}
}
}
}Redission red lock
The Redission red lock is actually an upgraded version of the above-mentioned redis distributed lock. The main reason is that the framework has already encapsulated the methods we need. In the actual process, we only need to import the corresponding jar package and use the corresponding API.
Maven引入：
org.redisson
redisson
3.17.4The pseudo code is as follows:
public JsonData getCoupon(long couponId, CouponCategoryEnum categoryEnum) {
String key = "lock:coupon:" + couponId;
RLock rLock = redisson.getLock(key);
LoginUser loginUser = LoginInterceptor.threadLocal.get();
rLock.lock();
try{
//业务逻辑
}finally {
rLock.unlock();
}
return JsonData.buildSuccess();
}In this way, there is no need to care about the renewal of the key expiration time, because once the lock is successfully locked in Redisson, a watch dog will be started. You can understand it as a daemon thread, which will default to every 30 seconds (flexible) Configuration) check, if the current client still holds the lock, it will automatically extend the expiration time of the lock.
Zookeeper distributed lock
Zookeeper distributed locks apply temporary sequential nodes. How to achieve it? Let's take a look at the detailed steps:
Acquire the lock:
First, create a persistent node ParentLock in Zookeeper. When the first client (Client1) wants to acquire the lock, it needs to create a temporary sequence node Lock1 under the ParentLock node.
Afterwards, Client1 searches and sorts all temporary sequential nodes under ParentLock, and judges whether the node Lock1 created by itself is the one with the highest sequence. If it is the first node, the lock is successfully acquired.
At this time, if another client Client2 comes to acquire the lock, a temporary sequential node Lock2 will be created in the ParentLock download.
Client2 searches and sorts all the temporary sequential nodes under ParentLock, and judges whether the node Lock2 created by itself is the one with the highest order, and finds that the node Lock2 is not the smallest.
Therefore, Client2 registers a Watcher with the node Lock1 whose ranking is just ahead of it, to monitor whether the Lock1 node exists. This means that Client2 failed to grab the lock and entered the waiting state.
At this time, if another client Client3 comes to acquire the lock, a temporary sequential node Lock3 is created in the ParentLock download.
Client3 searches and sorts all the temporary sequential nodes under ParentLock, and judges whether the node Lock3 created by itself is the one with the highest order, but also finds that the node Lock3 is not the highest.
Therefore, Client3 registers a Watcher with the node Lock2 whose ranking is just ahead of it, to monitor whether the Lock2 node exists. This means that Client3 also failed to grab the lock and entered the waiting state.
In this way, Client1 gets the lock, Client2 listens to Lock1, and Client3 listens to Lock2. This just forms a waiting queue, much like the AQS that ReentrantLock in Java relies on.
How to test for concurrency security issues?
First of all, we need to ensure that the project in the test environment is distributed and deployed in a cluster. Secondly, we can use the tool jmeter to concurrently request the coupon interface in the test environment according to the actual QPS of the coupon interface obtained online. After running for a period of time, we can go to the database again. Corresponding data, such as coupon inventory information, snap-up coupon information, etc., run it multiple times to see the effect.
Summarize
This article mainly shares a common concurrency security problem in e-commerce projects and the corresponding solution. From a performance point of view, it should be Redis > zookeeper > database. From the perspective of reliability (security): zookeeper > Redis > database.
Practical case
Optical theory is useless, you have to learn to follow along, and you have to do it yourself, so that you can apply what you have learned to practice. At this time, you can learn from some actual combat cases.
If it is helpful to you, please like and collect it to give the author an encouragement. It is also convenient for you to quickly find it next time.
If you don’t understand, please consult the small card below. The blogger also hopes to learn and progress with like-minded testers
At the right age, choose the right position, and try to give full play to your own advantages.
My road of automated test development is inseparable from the plan of each stage along the way, because I like planning and summarizing,
Test and develop video tutorials, study notes and receive portals! ! !