redis cache penetration, breakdown cache, the cache avalanche reason + Solutions

Watch meters pocket Java.md

I. Introduction

In our daily development, all are using the database to store data, due to the general system tasks typically do not exist in the case of high concurrency, so this seems to be no problem, but once the needs of large amount of data involved the system such as some commodity buying scenarios, or home visits moment when the larger, single-use database to store the data because of the speed-oriented disk, disk read / write slower problems serious performance drawbacks, the moment thousands of request comes in, the system needed to complete thousands of read / write operations in a very short period of time, this time more often than not be able to withstand the database, the database system is extremely easy to cause paralysis, eventually leading service downtime serious production problems.

In order to overcome the above problems, projects often introduce NoSQL technology, which is a memory-based database, and provide some persistence function.

redis NoSQL technology is a kind of art, but the introduction of redis there may appear to penetrate the cache, the cache breakdown, avalanche cache and other issues. In this paper, for a more in-depth analysis of these three issues.

Second, the beginning of understanding

Cache penetration: key corresponding data does not exist in the data source, each time a request for this not obtain the key from the cache, requests to the data source, which may overwhelm the data source. For example, with a non-existent user id for user information, regardless of database cache or not, if a hacker could exploit this vulnerability to attack overwhelmed the database.
Cache breakdown: key corresponding data exists, but the redis expired, over time if a large number of concurrent requests, these requests are usually found cache expiration DB and load data from the rear end to the back in the cache, this time a large concurrent requests It may momentarily overwhelmed by the backend DB.
Cache Avalanche: When the cache server reboot or a large number of cache concentrated in one time period fail, so that when failure, will bring a lot of pressure to the back-end systems (such as DB).

Third, the cache penetrating solution

A cache and can not find certain data does not exist, because the cache is not passive when writing a hit, and for fault-tolerant considerations, finding out if the data from the storage layer is not write cache, which will result in data that does not exist each request to the storage layer to make inquiries, lost the meaning of the cache.

There are many ways to effectively solve the problem of penetration of the cache , the most common is the use of a Bloom filter, all possible data hash to a sufficiently large bitmap in a certain absence of data will be the bitmap interception off, thus avoiding queries pressure on the underlying storage system. There is also a more simple and crude way (we use is this), if a query returned an empty data (whether data does not exist, or system failure), we still see the empty cache results, but it's expiration time will be very short, no longer than five minutes.

Brutal manner pseudocode:

//伪代码
public object GetProductListNew() {
    int cacheTime = 30;
    String cacheKey = "product_list";

    String cacheValue = CacheHelper.Get(cacheKey);
    if (cacheValue != null) {
        return cacheValue;
    }

    cacheValue = CacheHelper.Get(cacheKey);
    if (cacheValue != null) {
        return cacheValue;
    } else {
        //数据库查询不到，为空
        cacheValue = GetProductListFromDB();
        if (cacheValue == null) {
            //如果发现为空，设置个默认值，也缓存起来
            cacheValue = string.Empty;
        }
        CacheHelper.Add(cacheKey, cacheValue, cacheTime);
        return cacheValue;
    }
}

Fourth, the breakdown caching solutions

ultrahigh key may be accessed concurrently at certain time point, it is a very "hot spot" data. This time, it is necessary to consider the question: cache is "breakdown" of the problem.

Use mutex (mutex key)

More commonly used industry practice is to use a mutex. Briefly, that is, when a cache miss (decision is null out), it is not immediately to the load db, but the first caching tools used with some success in the operation returns to the operation value (such as the Redis SETNX or Memcache the ADD) to set a mutex key, when the operation returns successfully, then the operation load db and back in the cache; otherwise, you get to retry the entire cache method.

SETNX, is an abbreviation of "SET if Not eXists", that is, only when it does not exist is set, you can use it to achieve the effect of the lock.

public String get(key) {
      String value = redis.get(key);
      if (value == null) { //代表缓存值过期
          //设置3min的超时，防止del操作失败的时候，下次缓存过期一直不能load db
      if (redis.setnx(key_mutex, 1, 3 * 60) == 1) {  //代表设置成功
               value = db.get(key);
                      redis.set(key, value, expire_secs);
                      redis.del(key_mutex);
              } else {  //这个时候代表同时候的其他线程已经load db并回设到缓存了，这时候重试获取缓存值即可
                      sleep(50);
                      get(key);  //重试
              }
          } else {
              return value;      
          }
 }

memcache Code:

if (memcache.get(key) == null) {  
    // 3 min timeout to avoid mutex holder crash  
    if (memcache.add(key_mutex, 3 * 60 * 1000) == true) {  
        value = db.get(key);  
        memcache.set(key, value);  
        memcache.delete(key_mutex);  
    } else {  
        sleep(50);  
        retry();  
    }  
}

Other options: it is supplemented.

V. caching solutions avalanche

Breakdown cache difference is that here for many key cache, the former is one key.

Redis obtained from normal cache, the diagram is as follows:
redis1.md

Cache invalidation moment diagram is as follows:
redis2.md

缓存失效时的雪崩效应对底层系统的冲击非常可怕！大多数系统设计者考虑用加锁或者队列的方式保证来保证不会有大量的线程对数据库一次性进行读写，从而避免失效时大量的并发请求落到底层存储系统上。还有一个简单方案就时讲缓存失效时间分散开，比如我们可以在原有的失效时间基础上增加一个随机值，比如1-5分钟随机，这样每一个缓存的过期时间的重复率就会降低，就很难引发集体失效的事件。

加锁排队，伪代码如下：

//伪代码
public object GetProductListNew() {
    int cacheTime = 30;
    String cacheKey = "product_list";
    String lockKey = cacheKey;

    String cacheValue = CacheHelper.get(cacheKey);
    if (cacheValue != null) {
        return cacheValue;
    } else {
        synchronized(lockKey) {
            cacheValue = CacheHelper.get(cacheKey);
            if (cacheValue != null) {
                return cacheValue;
            } else {
              //这里一般是sql查询数据
                cacheValue = GetProductListFromDB(); 
                CacheHelper.Add(cacheKey, cacheValue, cacheTime);
            }
        }
        return cacheValue;
    }
}

加锁排队只是为了减轻数据库的压力，并没有提高系统吞吐量。假设在高并发下，缓存重建期间key是锁着的，这是过来1000个请求999个都在阻塞的。同样会导致用户等待超时，这是个治标不治本的方法！

注意：加锁排队的解决方式分布式环境的并发问题，有可能还要解决分布式锁的问题；线程还会被阻塞，用户体验很差！因此，在真正的高并发场景下很少使用！

随机值伪代码：

//伪代码
public object GetProductListNew() {
    int cacheTime = 30;
    String cacheKey = "product_list";
    //缓存标记
    String cacheSign = cacheKey + "_sign";

    String sign = CacheHelper.Get(cacheSign);
    //获取缓存值
    String cacheValue = CacheHelper.Get(cacheKey);
    if (sign != null) {
        return cacheValue; //未过期，直接返回
    } else {
        CacheHelper.Add(cacheSign, "1", cacheTime);
        ThreadPool.QueueUserWorkItem((arg) -> {
      //这里一般是 sql查询数据
            cacheValue = GetProductListFromDB(); 
          //日期设缓存时间的2倍，用于脏读
          CacheHelper.Add(cacheKey, cacheValue, cacheTime * 2);                 
        });
        return cacheValue;
    }
}

解释说明：

缓存标记：记录缓存数据是否过期，如果过期会触发通知另外的线程在后台去更新实际key的缓存；
缓存数据：它的过期时间比缓存标记的时间延长1倍，例：标记缓存时间30分钟，数据缓存设置为60分钟。这样，当缓存标记key过期后，实际缓存还能把旧数据返回给调用端，直到另外的线程在后台更新完成后，才会返回新缓存。

关于缓存崩溃的解决方法，这里提出了三种方案：使用锁或队列、设置过期标志更新缓存、为key设置不同的缓存失效时间，还有一种被称为“二级缓存”的解决方法。