[Redis | Black Horse Comments] Merchant Query Cache

What is caching?

Cache ( Cache) is the buffer for data exchange . The commonly known cache is the data in the buffer, which is generally obtained from the database and stored in the local code.It is a temporary place to store data, and generally has high read and write performance

In actual development, caching technology can prevent excessive data access from rushing to the system, causing its operating threads to be unable to process information in time and paralyzed;

In actual development, a multi-level cache will be built to further increase the speed of the system, for example: the local cache and the cache in redis are used concurrently

Browser cache : mainly the cache that exists on the browser side

Application layer cache : can be divided into tomcat local cache, such as the map mentioned before, or use redis as cache

Database cache : There is a piece of space in the database as a buffer pool, and the data added, modified, and checked will be loaded into the mysql cache first

CPU cache : The biggest problem of contemporary computers is that the CPU performance has improved, but the memory read and write speed has not kept up. Therefore, in order to adapt to the current situation, the CPU's L1, L2, and L3 caches have been added.

insert image description here
The role and cost of caching:

The cache data is stored in the code, and the code runs in the memory. The read and write performance of the memory is much higher than that of the disk. The cache can greatly reduce the read and write pressure on the server caused by the concurrent user access.
insert image description here

Add business cache

We can find that our merchant information is obtained from /shopthis interface, as shown in the figure below:
insert image description here
insert image description here

The following 1 is the id of this merchant.

Through the original code of the project, we can find that the merchant query here is a direct database:
insert image description here
then we will add a cache to this interface to improve query performance.

The standard operation method is to query the cache before querying the database. If the cached data exists, it will be returned directly from the cache. If the cached data does not exist, then query the database and store the data in redis.
insert image description here

According to this cache function model, we can roughly plan the process of querying store caches based on id:
insert image description here

We still put the logic in the Service layer, and then return it directly in the Controller.

The store information is an object. Our first reaction is to use the hash structure to store it, but it is also possible to use String. Here we use the latter.

In this place, it is recommended to start writing directly after getting the logic diagram, instead of watching the video and typing with the teacher. Because the logic of this place is similar to the previous business. I have a rough idea in mind, and the routines are similar:

  • You may need to use the hutool tool class
  • When saving to redis, the key must have a prefix

code show as below:

@Service
public class ShopServiceImpl extends ServiceImpl<ShopMapper, Shop> implements IShopService {
    
    

    @Autowired
    private StringRedisTemplate stringRedisTemplate;


    @Override
    public Result queryById(Long id) {
    
    
        //从redis中查询商铺缓存
        String shop = stringRedisTemplate.opsForValue().get(RedisConstants.CACHE_SHOP_KEY + id);
//        如果缓存命中则将json数据转换成对象返回
        if (StrUtil.isNotBlank(shop)) return Result.ok(JSONUtil.toBean(shop,Shop.class));
//        如果缓存没有命中则根据id查询数据库
        Shop shop1 = getById(id);
        //如果商户在数据库中都查询不到则直接报错404
        if (shop1 == null) return Result.fail("不存在此商户");
//        如果存在此商铺则将数据库中的商铺信息写回到redis中,然后再将信息返回
        stringRedisTemplate.opsForValue().set(RedisConstants.CACHE_SHOP_KEY + id,JSONUtil.toJsonStr(shop1));

        return Result.ok(shop1);

    }
}

When I wrote this code for the first time, the string was judged to be empty, and the logic of comparing it directly with null was not very strict. I took a look at the isNotBlank method of this tool class:
insert image description here
insert image description here
the judgment is more rigorous. In addition to judging the string is null, it also Null characters, spaces, etc. are considered.

Pay attention to the concept that
is easy to confuse here: we convert an object into Json and store it in redis, and then convert it from Json to object when we take it out of redis. This conversion is done manually by us. No one else does it for us.
In our Controller layer, the received Json is converted into an object, and the returned object is converted into Json. These actions rely on the built-in jackson in the SpringBoot framework, and we do not need to manually convert it ourselves.

Cache update strategy

Cache update is a thing designed by redis to save memory, mainly because the memory data is precious. When we insert too much data into redis, it may cause too much data in the cache, so redis will process some data Update, or it is more appropriate to call him eliminated.

Memory elimination: redis is automatically carried out. When the redis memory reaches the max-memery we set, it will automatically trigger the elimination mechanism to eliminate some unimportant data (you can set the strategy by yourself)

Timeout removal: When we set the expiration time ttl for redis, redis will delete the timed out data so that we can continue to use the cache

Active update: We can manually call the method to delete the cache, which is usually used to solve the problem of inconsistency between the cache and the database

insert image description here

Since the data source of our cache comes from the database , and the data in the database will change , if the data in the database changes, but the cache is not synchronized , there will be a consistency problem at this time , and the consequence is :
When users use outdated data in the cache, similar multi-threaded data security issues will arise, which will affect business, product reputation, etc.; how to solve it?

Active Update Policy

In order to maintain high consistency, we generally use the method of active update. This method has the following three strategies:

insert image description here

  • Cache Aside Pattern: The cache caller updates the cache after updating the database, also known as the double-write scheme ( manual coding method )

  • Read/Write Through Pattern: Completed by the system itself, the problem of database and cache is handled by the system itself ( with opacity, our operator does not know whether the operation is database or cache )

  • Write Behind Caching Pattern: The caller only operates the cache, and other threads process the database asynchronously to achieve 最终一致( cache write-back method ,That is, using the cache as the latest data source. Asynchronous processing can improve efficiency. For multiple operations in the cache, there may only be a small number of database operations after asynchronous processing. But its shortcomings are also very obvious, the data at both ends of the process is inconsistent, and if a downtime occurs, because the cache is in the memory, the data will be completely lost)

Solution 1 is more controllable, so the first manual coding solution is generally used in enterprises

When we use Option 1, we also need to consider the following three issues:

  • Delete cache or update cache?
  • How to ensure the simultaneous success and failure of cache and database operations?
  • Do you operate the cache first or the database first?
  • Delete cache or update cache?

    • Update cache: every time the database is updated, the cache is updated, and there are many invalid write operations
    • Delete cache: invalidate the cache when updating the database, and update the cache when querying
    • If the first solution is adopted, then suppose we operate the cache every time we operate the database, but if there is no query in the middle, then the update action will actually only take effect for the last time, and the update action in the middle is of little significance. We can put Cache deletion, when waiting for another query, load the data in the cache

  • How to ensure the success or failure of cache and database operations at the same time?

    • Monolithic system, put cache and database operations in one transaction
    • Distributed system, using distributed transaction schemes such as TCC
  • Do you operate the cache first or the database first?

    • Delete the cache first, and then operate the database
    • Operate the database first, then delete the cache

Should I specifically operate the cache or the database?

Option One:

  • normal circumstances:
  • insert image description here
  • abnormal situation:
  • insert image description here
  • When two threads come to access concurrently, assuming that thread 1 comes first, he deletes the cache first. At this time, thread 2 comes, and he queries that the cache data does not exist. At this time, he writes to the cache. After he writes to the cache, When thread 1 executes the update action again, the old data is actually written, and the new data is overwritten by the old data
  • Option 1 has a higher probability of abnormal situations, because it takes a long time to update the database in thread 1, and the query cache and write cache time in thread 2 are very short, so abnormal situations are still easy to occur.

Option II:

  • normal circumstances:
  • insert image description here
  • abnormal situation;
  • insert image description here
  • The probability of the abnormal situation of the second plan is extremely low

So in terms of thread safety, we generally choose option two

Summarize:
insert image description here

Realize that the store query cache is consistent with the database double-write

Modify the business logic in ShopController to meet the following requirements:

  • When querying a store based on the id, if the cache misses, query the database, write the database result to the cache, and set the timeout period (a bottom-up solution for timeout elimination)

  • When modifying a store based on id, modify the database first, then delete the cache

For requirement 1, we only need to modify the queryById method of ShopServiceImpl:
insert image description here

For requirement 2:
insert image description here
Then we can implement the updateByIdPlus method in ShopServiceImpl;
insert image description here

Database operations and cache operations advance and retreat at the same time, because it is a single system, so the Transactional annotation is used here.

Solution to cache penetration problem

缓存穿透: Cache penetration means that the data requested by the client does not exist in the cache or the database, so the cache will never take effect, and these requests will hit the database .

insert image description here

In such scenarios, all customer requests will reach the database. If this vulnerability is exploited, it will cause huge pressure on the database, or even be destroyed.

There are two common solutions:

  • 缓存空对象
    • Advantages: simple implementation, easy maintenance
    • shortcoming:
      • Additional memory consumption (TTL can be set to alleviate this problem)
      • May cause short-term inconsistencies
  • 布隆过滤
    • Advantages: less memory usage, no redundant keys
    • shortcoming:
      • achieve complex
      • There is a possibility of misjudgment

Analysis of the idea of ​​caching empty objects: When our client accesses non-existing data, it first requests redis, but there is no data in redis at this time, and the database will be accessed at this time, but there is no data in the database, and this data penetrates the cache. Going straight to the database, we all know that the concurrency that the database can carry is not as high as that of redis. If a large number of requests come to access this kind of non-existent data at the same time, these requests will all access the database. The simple solution is that even if the data is in the database. Does not exist, we also store this data in redis, so that next time the user comes to access the non-existent data, then the data can be found in redis and will not enter the cache

Bloom filter: Bloom filter actually uses the hash idea to solve this problem. Through a huge binary array, use the hash idea to judge whether the current data to be queried exists. If the Bloom filter judges that it exists , then release, this request will go to redis, even if the data in redis has expired at this time, but this data must exist in the database, after querying the data in the database, put it into redis,

Assuming that the Bloom filter judges that the data does not exist, it returns directly

The advantage of this method is that it saves memory space, and there is a misjudgment. The reason for the misjudgment is that the Bloom filter uses the hash idea. As long as the hash idea is used, there may be hash conflicts.

Bloom filter schematic:
insert image description here

insert image description here

The left picture shows the principle of caching empty objects, and the right picture shows the principle of Bloom filtering.

Coding solves the cache penetration problem of commodity queries

Here we use the idea of ​​caching empty objects to solve this problem

The core idea is as follows:

In the original logic, if we find that this data does not exist in mysql, we will return 404 directly, so there will be a problem of cache penetration

In the current logic: if this data does not exist, we will not return 404, but will still write this data to Redis, and set the value to empty. When the query is initiated again, if we find a hit, we will judge this Whether the value is null, if it is null, it is the data written before, which proves to be the cache penetration data, if not, the data will be returned directly.

insert image description here

Here we modify the queryById method in ShopServiceImpl, the code is as follows:

	@Override
    public Result queryById(Long id) {
    
    
        //从redis中查询商铺缓存
        String shop = stringRedisTemplate.opsForValue().get(RedisConstants.CACHE_SHOP_KEY + id);
//        如果缓存命中则将json数据转换成对象返回
        if (StrUtil.isNotBlank(shop)) return Result.ok(JSONUtil.toBean(shop,Shop.class));
        //然后我们要处理缓存穿透的空值情况
        if(shop != null) return Result.fail("店铺信息不存在");
//        如果缓存没有命中则根据id查询数据库
        Shop shop1 = getById(id);
        //如果商户在数据库中都查询不到则直接报错404(废弃)
//        if (shop1 == null) return Result.fail("不存在此商户");
        //如果商户在数据库中查询不到则将空值写入redis,预防缓存击穿的情况
        if (shop1 == null) {
    
    
            stringRedisTemplate.opsForValue().set(RedisConstants.CACHE_SHOP_KEY + id,"",RedisConstants.CACHE_NULL_TTL,TimeUnit.MINUTES);
            Result.fail("不存在此商户");
        }
//        如果存在此商铺则将数据库中的商铺信息写回到redis中,然后再将信息返回
        stringRedisTemplate.opsForValue().set(RedisConstants.CACHE_SHOP_KEY + id,JSONUtil.toJsonStr(shop1),RedisConstants.CACHE_SHOP_TTL, TimeUnit.MINUTES);

        return Result.ok(shop1);

    }

When we deal with the null value of cache penetration, shop != nullsome people may not understand why they wrote it in the first place.

We must first know that the shop we found from the cache has three values:

  • ""
  • null
  • json value

After StrUtil.isNotBlank(shop)one layer of filtering, only the nullsum ""is left, and if we directly use the equals method to judge whether it is "". There will be a null pointer, so we use hereshop != null

Small summary:

What is the cause of cache penetration?

  • The data requested by the user does not exist in the cache or in the database, and such requests are continuously initiated, which brings huge pressure to the database

What are the solutions for cache penetration?

  • Cache null values
  • bloom filter
  • Enhance the complexity of id to avoid guessing id rules
  • Do a good job of data basic format verification
  • Strengthen user permission verification
  • Do a good job of limiting the hotspot parameters

Cache avalanche problem and solution

缓存雪崩: It means that a large number of cache keys fail at the same time or the Redis service goes down, causing a large number of requests to reach the database, bringing huge pressure.

solution:

  • Add a random value to the TTL of different Keys
    • When warming up the cache, the data in the database will be imported into the cache in advance. At this time, there will be a large number of keys with the same TTL, and they will fail at a certain point in time, causing an avalanche. And if we add random values ​​to the TTL of different keys, these keys will be invalidated within a period of time, which greatly reduces the occurrence of cache avalanche.
  • Using Redis Cluster to Improve Service Availability
    • sentinel mechanism
  • Add a degraded current limiting policy to the cache business
    • That is to sacrifice some services (deny access) to protect the health of the database
  • Add multi-level cache to business

insert image description here

Cache breakdown problem and solution

缓存击穿The problem is also called the hotspot key problem. It is a key that is accessed by high concurrency and the cache reconstruction business is complicated . The key suddenly becomes invalid. Countless access requests will bring a huge impact to the database in an instant.

There are two common solutions:

  • mutex
  • logical expiration

Logical analysis: Assume that thread 1 should query the database after querying the cache, and then reload the data to the cache. At this time, as long as thread 1 completes this logic, other threads can load the data from the cache. But suppose that when thread 1 is not finished, subsequent thread 2, thread 3, and thread 4 come to access the current method at the same time, then these threads cannot query data from the cache, then they will access the query cache at the same time , did not find it, and then access the database at the same time, and execute the database code at the same time, which puts too much pressure on database access

insert image description here

Solution 1: Use a lock

Because locks can achieve mutual exclusion. Assuming that the thread comes over, only one person can access the database, so as to avoid excessive pressure on database access, but this will also affect the performance of the query, because at this time, the performance of the query will change from parallel to serial. We You can use the tryLock method + double check to solve such problems.

Assuming that thread 1 comes to visit now, he does not hit the query cache, but at this time he obtains the resource of the lock, then thread 1 will execute the logic alone, assuming that thread 2 comes now, thread 2 has not obtained the lock resource during execution. When the lock is found, thread 2 can go to sleep until thread 1 releases the lock, thread 2 obtains the lock, and then executes the logic. At this time, the data can be obtained from the cache.

insert image description here

Solution 2: Logical expiration scheme

Solution analysis: The main reason why we have this cache breakdown problem is that we have set an expiration time for the key. If we do not set an expiration time, there will be no cache breakdown problem, but if we do not set an expiration time, Doesn’t this data always occupy our memory? We can adopt a logical expiration scheme.

We set the expiration time in the value of redis. Note: this expiration time will not directly affect redis, but we will deal with it through logic later. Suppose thread 1 goes to query the cache, and then judges from the value that the current data has expired. At this time, thread 1 goes to obtain the mutex, then other threads will block, and the thread that has obtained the lock will open a thread to perform the previous The logic of reconstructing data does not release the lock until the newly opened thread completes this logic, and thread 1 returns directly. Suppose now that thread 3 comes to visit, because thread 2 holds the lock, thread 3 cannot acquire the lock , thread 3 also directly returns the data, and only after the newly opened thread 2 has constructed the reconstructed data can other threads return the correct data.

The ingenuity of this solution lies in the fact that the cache is built asynchronously. The disadvantage is that all dirty data is returned before the cache is built.

insert image description here

comparing

Mutual exclusion lock scheme: Because mutual exclusion is guaranteed, the data is consistent and the implementation is simple, because only one lock is needed, and there are no other things to worry about, so there is no additional memory consumption. The disadvantage is that there is no lock. There is a deadlock problem, and only serial execution performance will definitely be affected

Logical expiration scheme: There is no need to wait during the thread reading process, and the performance is good. There is an additional thread holding a lock to reconstruct the data, but before the reconstructed data is completed, other threads can only return the previous data, and realize up trouble

insert image description here

Solve the problem of cache breakdown based on mutex

Requirements: Modify the business of querying stores based on id, and solve the problem of cache breakdown based on mutex locks

Core idea: Compared with querying the database directly after querying no data from the cache, the current solution is to acquire a mutex if no data is found from the cache after querying. After acquiring the mutex , to determine whether the lock has been obtained, if not obtained, sleep, and try again after a while, until the lock is obtained, the query can be performed

If you get the locked thread, then go to query, write the data into redis after the query, release the lock, return the data, and use the mutex to ensure that only one thread executes the logic of operating the database to prevent cache breakdown

insert image description here

Note:
The lock here is not the synchronized or lock we usually use. It can be executed when the lock is obtained, and waits if the lock is not obtained. And we need to customize the execution logic of getting the lock and not getting the lock here, so we can't use the original methods. So what method do we use to customize this mutex (that is to say, only one thread of multiple threads can succeed in parallel, and other threads can fail)?
In the string type of redis, we have come into contact with a command setnx:
insert image description here
that is equivalent to:

  • To acquire a lock is to assign a value with setnx
  • To release the lock is to delete the key

When we use setnx to add keys, we usually set a limited period to prevent the lock from being released due to some unexpected failures.

First of all, we first write two methods to represent the acquisition and release of the lock, which is convenient for later use in the business:

    //获取锁
    private boolean tryLock(String key){
    
    
        Boolean flag = stringRedisTemplate.opsForValue().setIfAbsent(key, "1", 10, TimeUnit.SECONDS);
        return BooleanUtil.isTrue(flag);
    }
    //释放锁
    private void unLock(String key){
    
    
        stringRedisTemplate.delete(key);
    }

Note that BooleanUtil.isTrue is used here because our flag is Boolean and the method returns boolean, so it will be automatically unboxed here, and the essence of automatic unboxing is to execute the valueOf method corresponding to the packaging class. If the flag is null, it will appear The null pointer case.

The overall code is as follows:

    /**
     * 缓存击穿使用互斥锁解决
     * @param id
     * @return
     */
    public Shop queryWithMutex(Long id){
    
    
        // 1、从redis中查询商铺缓存
        String shopJson = stringRedisTemplate.opsForValue().get(RedisConstants.CACHE_SHOP_KEY + id);
        // 2、判断是否存在
        if (StrUtil.isNotBlank(shopJson)) {
    
    
            // 存在,直接返回
            return JSONUtil.toBean(shopJson, Shop.class);
        }
        //判断命中的值是否是空值
        if (shopJson != null) {
    
    
            //返回一个错误信息
            return null;
        }
        // 4.实现缓存重构
        //4.1 获取互斥锁
        String lockKey = RedisConstants.LOCK_SHOP_KEY + id;
        Shop shop = null;
        try {
    
    
            boolean isLock = tryLock(lockKey);
            // 4.2 判断否获取成功
            if(!isLock){
    
    
                //4.3 失败,则休眠重试
                Thread.sleep(50);
                return queryWithMutex(id);
            }
            //4.4 成功,根据id查询数据库
            shop = getById(id);
            // 5.不存在,返回错误
            if(shop == null){
    
    
                //将空值写入redis
                stringRedisTemplate.opsForValue().set(RedisConstants.CACHE_SHOP_KEY + id,"",CACHE_NULL_TTL,TimeUnit.MINUTES);
                //返回错误信息
                return null;
            }
            //6.写入redis
            stringRedisTemplate.opsForValue().set(RedisConstants.CACHE_SHOP_KEY + id,JSONUtil.toJsonStr(shop),CACHE_NULL_TTL,TimeUnit.MINUTES);

        }catch (Exception e){
    
    
            throw new RuntimeException(e);
        }
        finally {
    
    
            //7.释放互斥锁
            unLock(lockKey);
        }
        return shop;
    }

Solve the problem of cache breakdown based on logical expiration

Requirement: Modify the business of querying stores based on id, and solve the problem of cache breakdown based on logical expiration

Idea analysis: When the user starts to query redis, it is judged whether it is a hit. If there is no hit, the empty data is returned directly without querying the database. Once it is hit, the value is taken out to determine whether the expiration time in the value is satisfied. If it is not expired, then Directly return the data in redis. If it expires, try to acquire the lock. If the lock is not acquired, it means that someone is rebuilding the cache now, so directly return the dirty data. If the lock is acquired, open the independent thread and return the previous data directly. Independently The thread reconstructs the data, and releases the mutex after the reconstruction is completed.

insert image description here

Since the data we store in redis needs an additional expiration time, here we have two processing methods:

  • Create a new class, give it an expiration time attribute, and let the shop class inherit it
  • Create a new class, assign the expiration time attribute and the data attribute of the Object class, and store it directly as a new entity class

The previous method caused changes to the source code and is not recommended. The latter code is less intrusive to the code, and in later development, other classes except the shop class can also use this for data storage.

Step 1: Create RedisData

@Data
public class RedisData {
    
    
    private LocalDateTime expireTime;
    private Object data;
}

Step 2: Simulate the cache warm-up scenario

In actual situations, we must first cache and warm up these hot keys in the background before doing activities. Here we add a new method in ShopServiceImpl , and then use the unit test to warm up the cache.

Cache warming is divided into three steps:

  • Query store data
  • Encapsulate data and logical expiration time
  • write to redis

code show as below:

In ShopServiceImpl:

    public void preSaveShopInRedis(Long id,Long expireSeconds){
    
    
        //查询店铺信息
        Shop shop = getById(id);
        //封装数据,并且设置过期时间
        RedisData container = new RedisData();
        container.setData(shop);
        container.setExpireTime(LocalDateTime.now().plusSeconds(expireSeconds));
        //写入到redis中
        stringRedisTemplate.opsForValue().set(RedisConstants.CACHE_SHOP_KEY + id,JSONUtil.toJsonStr(container));
    }

test:

insert image description here
result:

insert image description here
After completing the first two steps, we officially start writing business code:

There are two main issues here:

The object we took out from redisData is of type Object, and what we want is of type shop. We know that the deserialization of Json depends on bytecode ( .classfile):
insert image description here
but in our bytecode file, it is indicated that data is of type Object:
insert image description here
when we use the getData method of redisData, we get an Object type, But its essence is actually the JSONObject type. We can use this type and then use the JSONUtil.toBean method to provide the corresponding bytecode file and convert it into the type we want.

Another problem is to create a new thread. We know that there are four ways to create a new thread:

  • Inherit the Thread class to create threads

  • Implement the Runnable interface to create threads

  • Create threads using Callable and Future

  • Use a thread pool such as the Executor framework

Here we use the method of thread pool. The specific method is to use Executors to create a thread pool with a fixed number of threads ( FixedThreadPool)

The complete code of the business is as follows:

    public static final ExecutorService CACHE_REBUILD_EXECUTOR = Executors.newFixedThreadPool(10);
    /**
     * 缓存击穿使用逻辑过期时间解决
     * @param id
     * @return
     */
    public Shop queryWithLogic(Long id){
    
    
        //从redis中查询商铺缓存
        String box = stringRedisTemplate.opsForValue().get(RedisConstants.CACHE_SHOP_KEY + id);
//        如果缓存未命中则直接返回空
        if (StrUtil.isBlank(box)) return null;
//        如果缓存命中则先查看缓存是否过期
        RedisData redisData = JSONUtil.toBean(box, RedisData.class);
        Shop shop = JSONUtil.toBean((JSONObject) redisData.getData(), Shop.class);
        LocalDateTime expireTime = redisData.getExpireTime();
        if (expireTime.isAfter(LocalDateTime.now())){
    
    
            //说明没有过期。这个时候直接返回脏数据
            return shop;
        }
        //如果过期了,尝试获取互斥锁
        boolean lock = tryLock(RedisConstants.LOCK_SHOP_KEY + id);
        if (lock){
    
    
            //如果锁获取成功了,开启新的线程
            CACHE_REBUILD_EXECUTOR.submit(() -> {
    
    
                try {
    
    
                    this.preSaveShopInRedis(id,20L);
                } catch (Exception e) {
    
    
                    e.printStackTrace();
                } finally {
    
    
                    unLock(RedisConstants.LOCK_SHOP_KEY + id);
                }
            });
        }
        return shop;
    }

Note:
①The submit method requires an instance that implements the runnable interface, and the runnable interface is a single-method interface, so we can directly use lambda expressions or method references.
②In order to ensure the lock release problem caused by unnecessary accidents, we generally write the lock release in finally.
③Comparing the sequence of time, we use the isAfter method.

Encapsulate redis tool class

Through the previous content, we can find that their code logic is somewhat complicated whether it is to solve cache penetration or cache breakdown. If we write these logics every time we develop, the cost of development will become very high, so we generally package these solutions into tools for subsequent use.

Encapsulate a cache tool class based on StringRedisTemplate to meet the following requirements:

  • 方法1: Serialize any Java object into json and store it in the key of string type, and you can set the TTL expiration time
  • 方法2: Serialize any Java object into json and store it in a key of string type, and you can set a logical expiration time to deal with the problem of cache breakdown
  • 方法3: Query the cache according to the specified key, and deserialize it into the specified type, and use the cache null value to solve the cache penetration problem
  • 方法4: Query the cache according to the specified key and deserialize it into the specified type. It is necessary to use logical expiration to solve the problem of cache breakdown
@Slf4j
@Component
public class CacheClient {
    
    

    private final StringRedisTemplate stringRedisTemplate;

    private static final ExecutorService CACHE_REBUILD_EXECUTOR = Executors.newFixedThreadPool(10);

    public CacheClient(StringRedisTemplate stringRedisTemplate) {
    
    
        this.stringRedisTemplate = stringRedisTemplate;
    }

    public void set(String key, Object value, Long time, TimeUnit unit) {
    
    
        stringRedisTemplate.opsForValue().set(key, JSONUtil.toJsonStr(value), time, unit);
    }

    public void setWithLogicalExpire(String key, Object value, Long time, TimeUnit unit) {
    
    
        // 设置逻辑过期
        RedisData redisData = new RedisData();
        redisData.setData(value);
        redisData.setExpireTime(LocalDateTime.now().plusSeconds(unit.toSeconds(time)));
        // 写入Redis
        stringRedisTemplate.opsForValue().set(key, JSONUtil.toJsonStr(redisData));
    }

    public <R,ID> R queryWithPassThrough(
            String keyPrefix, ID id, Class<R> type, Function<ID, R> dbFallback, Long time, TimeUnit unit){
    
    
        String key = keyPrefix + id;
        // 1.从redis查询商铺缓存
        String json = stringRedisTemplate.opsForValue().get(key);
        // 2.判断是否存在
        if (StrUtil.isNotBlank(json)) {
    
    
            // 3.存在,直接返回
            return JSONUtil.toBean(json, type);
        }
        // 判断命中的是否是空值
        if (json != null) {
    
    
            // 返回一个错误信息
            return null;
        }

        // 4.不存在,根据id查询数据库
        R r = dbFallback.apply(id);
        // 5.不存在,返回错误
        if (r == null) {
    
    
            // 将空值写入redis
            stringRedisTemplate.opsForValue().set(key, "", CACHE_NULL_TTL, TimeUnit.MINUTES);
            // 返回错误信息
            return null;
        }
        // 6.存在,写入redis
        this.set(key, r, time, unit);
        return r;
    }

    public <R, ID> R queryWithLogicalExpire(
            String keyPrefix, ID id, Class<R> type, Function<ID, R> dbFallback, Long time, TimeUnit unit) {
    
    
        String key = keyPrefix + id;
        // 1.从redis查询商铺缓存
        String json = stringRedisTemplate.opsForValue().get(key);
        // 2.判断是否存在
        if (StrUtil.isBlank(json)) {
    
    
            // 3.存在,直接返回
            return null;
        }
        // 4.命中,需要先把json反序列化为对象
        RedisData redisData = JSONUtil.toBean(json, RedisData.class);
        R r = JSONUtil.toBean((JSONObject) redisData.getData(), type);
        LocalDateTime expireTime = redisData.getExpireTime();
        // 5.判断是否过期
        if(expireTime.isAfter(LocalDateTime.now())) {
    
    
            // 5.1.未过期,直接返回店铺信息
            return r;
        }
        // 5.2.已过期,需要缓存重建
        // 6.缓存重建
        // 6.1.获取互斥锁
        String lockKey = LOCK_SHOP_KEY + id;
        boolean isLock = tryLock(lockKey);
        // 6.2.判断是否获取锁成功
        if (isLock){
    
    
            // 6.3.成功,开启独立线程,实现缓存重建
            CACHE_REBUILD_EXECUTOR.submit(() -> {
    
    
                try {
    
    
                    // 查询数据库
                    R newR = dbFallback.apply(id);
                    // 重建缓存
                    this.setWithLogicalExpire(key, newR, time, unit);
                } catch (Exception e) {
    
    
                    throw new RuntimeException(e);
                }finally {
    
    
                    // 释放锁
                    unlock(lockKey);
                }
            });
        }
        // 6.4.返回过期的商铺信息
        return r;
    }

    public <R, ID> R queryWithMutex(
            String keyPrefix, ID id, Class<R> type, Function<ID, R> dbFallback, Long time, TimeUnit unit) {
    
    
        String key = keyPrefix + id;
        // 1.从redis查询商铺缓存
        String shopJson = stringRedisTemplate.opsForValue().get(key);
        // 2.判断是否存在
        if (StrUtil.isNotBlank(shopJson)) {
    
    
            // 3.存在,直接返回
            return JSONUtil.toBean(shopJson, type);
        }
        // 判断命中的是否是空值
        if (shopJson != null) {
    
    
            // 返回一个错误信息
            return null;
        }

        // 4.实现缓存重建
        // 4.1.获取互斥锁
        String lockKey = LOCK_SHOP_KEY + id;
        R r = null;
        try {
    
    
            boolean isLock = tryLock(lockKey);
            // 4.2.判断是否获取成功
            if (!isLock) {
    
    
                // 4.3.获取锁失败,休眠并重试
                Thread.sleep(50);
                return queryWithMutex(keyPrefix, id, type, dbFallback, time, unit);
            }
            // 4.4.获取锁成功,根据id查询数据库
            r = dbFallback.apply(id);
            // 5.不存在,返回错误
            if (r == null) {
    
    
                // 将空值写入redis
                stringRedisTemplate.opsForValue().set(key, "", CACHE_NULL_TTL, TimeUnit.MINUTES);
                // 返回错误信息
                return null;
            }
            // 6.存在,写入redis
            this.set(key, r, time, unit);
        } catch (InterruptedException e) {
    
    
            throw new RuntimeException(e);
        }finally {
    
    
            // 7.释放锁
            unlock(lockKey);
        }
        // 8.返回
        return r;
    }

    private boolean tryLock(String key) {
    
    
        Boolean flag = stringRedisTemplate.opsForValue().setIfAbsent(key, "1", 10, TimeUnit.SECONDS);
        return BooleanUtil.isTrue(flag);
    }

    private void unlock(String key) {
    
    
        stringRedisTemplate.delete(key);
    }
}

Then we go to ShopServiceImpl to use our encapsulated redis tool class:

@Resource
private CacheClient cacheClient;

 @Override
    public Result queryById(Long id) {
    
    
        // 解决缓存穿透
        Shop shop = cacheClient
                .queryWithPassThrough(CACHE_SHOP_KEY, id, Shop.class, this::getById, CACHE_SHOP_TTL, TimeUnit.MINUTES);

        // 互斥锁解决缓存击穿
        // Shop shop = cacheClient
        //         .queryWithMutex(CACHE_SHOP_KEY, id, Shop.class, this::getById, CACHE_SHOP_TTL, TimeUnit.MINUTES);

        // 逻辑过期解决缓存击穿
        // Shop shop = cacheClient
        //         .queryWithLogicalExpire(CACHE_SHOP_KEY, id, Shop.class, this::getById, 20L, TimeUnit.SECONDS);

        if (shop == null) {
    
    
            return Result.fail("店铺不存在!");
        }
        // 7.返回
        return Result.ok(shop);
    }

Guess you like

Origin blog.csdn.net/zyb18507175502/article/details/127897144