Redis cache penetration and cache avalanche and solutions

Redis cache penetration and solutions

1. Cache penetration

1. When the key queried by the user does not exist in redis, and the corresponding id does not exist in the database, when an illegal user is attacked, a large number of requests will be directly hit on the db, causing downtime, which affects the entire system. This phenomenon is called cache penetration.

2. Solution 1: Cache empty data, such as empty string, empty object, empty array or list, the code is as follows

if (list != null && list.size() > 0) {
                redisOperator.set("subCat:" + rootCatId, JsonUtils.objectToJson(list));
            } else {
                redisOperator.set("subCat:" + rootCatId, JsonUtils.objectToJson(list), 5*60);
            }

3. Solution 2: Bloom filter

Bloom filter: judge whether an element is in an array, as shown in the figure below, a storage made by binary system, takes up relatively small memory, 0 means no existence, 1 means existence, adding query efficiency is very fast, when a value is saved Will go through an algorithm to save the corresponding value to a certain position on the set of bloom filters. There may be multiple keys in a certain position. When a non-existent key value is passed in, it will be matched with the set. If it matches If not, it will return a null.
Disadvantages: 1. 1% misjudgment rate, when a key does not exist in the Bloom array, but due to this misjudgment rate, it will be judged that the key exists under certain circumstances, when the array The longer the length, the lower the false positive rate, and the shorter the array, the higher the false positive rate.
2. When we want to delete a key value, we will delete the content in our database and redis, but the Bloom array cannot be deleted because There will be a pair of keys in a certain position of the array. If we want to delete it, we will change 1 to 0, but all the key values ​​in it will be deleted
. 3. The code complexity will increase because we have to maintain an additional collection. , When we use redis cluster, bloom filter should be used in combination with redis

Two, Redis cache avalanche

1. Cache avalanche: The data in the cache is invalidated in large quantities, and then a large number of requests come in for this use, but because the keys in redis are all invalidated, all requests will be sent to the db, causing downtime

2. Solution

1. Set the corresponding hot key to never expire
2. The expiration time is staggered, the expiration time is randomly generated, and the expiration time of the hot data can be set longer, and the non-hot data can be set shorter
3. Combination of multiple caches, such as: request entry, You can request redis now, when redis does not exist, then request memcache, if not, then request db
4. Procurement of third-party Redis (redis on Alibaba Cloud or Tencent Cloud)