How is Spring caching implemented? How to extend it to support expire delete function?

Preface: In our application, some data is remote data obtained through rpc. This data does not change frequently, allowing the client to cache locally for a certain period of time.

The logic of this scenario is simple, the cached data is small, and does not need to be persisted. Therefore, it is not desirable to introduce other third-party caching tools to increase the burden on the application. It is very suitable to use Spring Cache to implement.

But there is a problem that we want to cache these rpc result data and delete it automatically after a certain period of time, so as to obtain the latest data after a certain period of time. Similar to the expiration time of Redis.
What follows is my research steps and development process.

What is Spring Cache?

Spring Cache is a caching abstraction layer of Spring. Its function is to automatically cache the returned results when the method is called to improve system performance and response speed.
The goal is to simplify the use of caches, provide a consistent way to access caches, and enable developers to easily and quickly add caches to applications.
Applied at the method level, the next time a method with the same parameters is called, the result is fetched directly from the cache without having to execute the actual method body.

Applicable scene?

including but not limited to:

Frequently accessed method calls can improve performance by caching the results
Database query results, query results can be cached to reduce database access
External service call results, the response results of external services can be cached to reduce network overhead
Calculation results, which can be cached to speed up subsequent calculations

Advantages and disadvantages

advantage:

Improve application performance by avoiding repeated calculations or queries.
Reduce the load by reducing access to underlying resources, such as databases or remote services.
Simplify the code and implement the caching logic through annotations without manually writing caching code.

shortcoming:

A certain amount of memory space is required to store cached data.
It may cause data inconsistency. If the cached data changes but the cache is not updated in time, it may cause dirty data. (so the cache needs to be updated in time)
It may cause cache penetration problems. When a large number of requests access a key that does not exist in the cache at the same time, it will cause the requests to fall directly to the underlying resources and increase the load.

important components

CacheManager: Cache manager for creating, configuring and managing cache objects. You can configure specific cache implementations, such as Ehcache and Redis.
Cache: A cache object, used to store cached data, provides methods for reading, writing, and deleting cached data.
Common annotations:
- @Cacheable: When called, it will check whether it already exists in the cache. If so, the cached result will be returned directly. Otherwise, the method will be executed and the result will be stored in the cache, which is suitable for read-only operations.
- @CachePut: The method body will be executed every time, and the result will be stored in the cache, that is, the data in the cache will be updated every time, which is suitable for write operations.
- @CacheEvict: When called, Spring Cache will clear the corresponding cached data.

How to use

Configure the cache manager (CacheManager): Use @EnableCachingannotations to enable caching and configure specific cache implementations.
Add cache annotations to methods: use @Cacheable, @CacheEvict, @CachePutand other annotations to mark methods that need to be cached.
Calling a cached method: When calling a method that is marked as cached, Spring Cache will check whether the cached result of the method is already in the cache.
Return data according to the cached result: if there is a result in the cache, return it directly from the cache; otherwise, execute the method and store the result in the cache.
Clear or update the cache as needed: Using the annotation @CacheEvict, @CachePutthe cache can be cleared or updated after a method call.
Through the above steps, Spring Cache can automatically manage the read and write operations of the cache, thus simplifying the use and management of the cache.

Which implementation does Spring Boot use by default, and its pros and cons:

Spring Boot uses ConcurrentMapCacheManagerthe cache manager implementation by default, which is suitable for simple, stand-alone application scenarios that require less cache capacity.

advantage:
1. Simple and lightweight: no external dependencies, suitable for simple application scenarios.
2. Memory storage: Cache data is stored in memory ConcurrentMap, with fast read and write speeds, suitable for fast access and frequently updated data.
3. Multi-cache instance support: supports the configuration of multiple named cache instances, each instance uses independent ConcurrentMapstorage data, and multiple cache instances can be configured according to different needs.
shortcoming:
1. Stand-alone application limitations: ConcurrentMapCacheManagerApplicable to stand-alone applications, cached data is stored in the application's memory, and distributed caching cannot be implemented.
2. Limited capacity: Since the cached data is stored in the memory, ConcurrentMapCacheManagerthe capacity is limited by the memory size of the application. There may be a problem of insufficient capacity for large-scale data or high concurrent access scenarios.
3. Lack of persistence support: ConcurrentMapCacheManagerPersistence of cached data to disk or other external storage media is not supported, and the cached data will be lost after the application is restarted.

How to let `ConcurrentMapCacheManager`the support expiration be automatically deleted

As mentioned in the preface, our scenario logic is simple, the cached data is small, and does not require persistence. We do not want to introduce other third-party caching tools to increase the burden on the application, so it is suitable for use ConcurrentMapCacheManager. So the extension ConcurrentMapCacheManagermay be the simplest implementation.

Design

To this end, I designed three options:

Start the scheduled task, scan the cache, and delete all caches regularly; this method is simple and rude, and it is deleted at a unified time, but it cannot be expired for a single piece of data.
Start the scheduled task, scan the cache, and delete a single expired cache data.
Before accessing the cached data, judge whether it is expired, if expired, re-execute the method body, and overwrite the original cached data with the result.

The above solutions 2 and 3 are closer to the goal, and they all have a common difficulty, that is, how to judge whether the cache has expired? Or how to store the expiration time of the cache?
Since there is no good way, let's go to the source code to find ideas!

Source code analysis

ConcurrentMapCacheManagercacheMapA (code below) is defined in , which is used to store all cache names and corresponding cache objects.

private final ConcurrentMap<String, Cache> cacheMap = new ConcurrentHashMap<>(16);

cacheMapCacheThe specific type of storage in is ConcurrentMapCache,
and ConcurrentMapCachean internal definition store(the following code) is used to store all keys and values in the cache, that is, the real cache data.

private final ConcurrentMap<Object, Object> store;

Its relationship diagram is:

The following is the test code to add a cache operation for a query: cacheName=getUsersByName, key is the value of the parameter name, and value is the set of query users.

@Service
public class UserServiceImpl implements UserService {
    @Autowired
    private UserMapper userMapper;

    @Override
    @Cacheable(value = "getUsersByName", key = "#name")
    public List<GyhUser> getUsersByName(String name) {
        return userMapper.getUsersByName(name);
    }
}

Before the program calls this method, it will automatically enter the cache interceptor CacheInterceptor, and then enter ConcurrentMapCacheManagerthe getCachemethod to obtain the corresponding cache instance. If it does not exist, one will be generated.

Then look up the cached data from the cache instance, return if found, and execute the target method if not found.

After the target method is executed, the returned result is placed in the cache.

Implement automatic expiration deletion

According to the above code tracking, it can be found that the cache data key/value is stored in the specific cache instance ConcurrentMapCache, storeand there is room for me to operate before and after get and put.

So, if I repackage the value, encapsulate the cache time, and parse out the real cache data before and after get and put for developers to use, is it possible? Just do it!

/**
 * 缓存数据包装类，保证缓存数据及插入时间
 */
public class ExpireCacheWrap {
    /**
     * 缓存数据
     */
    private final Object value;
    /**
     * 插入时间
     */
    private final Long insertTime;

    public ExpireCacheWrap(Object value, Long insertTime) {
        this.value = value;
        this.insertTime = insertTime;
    }

    public Object getValue() {
        return value;
    }

    public Long getInsertTime() {
        return this.insertTime;
    }
}

Customize a Cacheclass, inherit ConcurrentMapCache, extend the get and put methods, and realize the recording and analysis of the cache time

/**
 * 缓存过期删除
 */
public class ExpireCache extends ConcurrentMapCache {
    public ExpireCache(String name) {
        super(name);
    }

    @Override
    public ValueWrapper get(Object key) {
        // 解析缓存对象时，拿到value，去掉插入时间。对于业务中缓存的使用逻辑无感知无侵入，无需调整相关代码
        ValueWrapper valueWrapper = super.get(key);
        if (valueWrapper == null) {
            return null;
        }
        Object storeValue = valueWrapper.get();
        storeValue = storeValue != null ? ((ExpireCacheWrap) storeValue).getValue() : null;
        return super.toValueWrapper(storeValue);
    }

    @Override
    public void put(Object key, @Nullable Object value) {
        // 插入缓存对象时，封装对象信息：缓存内容+插入时间
        value = new ExpireCacheWrap(value, System.currentTimeMillis());
        super.put(key, value);
    }
}

Customize the cache manager, ExpireCachereplace the default one with the custom oneConcurrentMapCache

/**
 * 缓存管理器
 */
public class ExpireCacheManager extends ConcurrentMapCacheManager {
    @Override
    protected Cache createConcurrentMapCache(String name) {
        return new ExpireCache(name);
    }
}

Inject a custom cache manager ExpireCacheManagerinto the container

@Configuration
class ExpireCacheConfiguration {
    @Bean
    public ExpireCacheManager cacheManager() {
        ExpireCacheManager cacheManager = new ExpireCacheManager();
        return cacheManager;
    }
}

Turn on scheduled tasks and automatically delete expired caches

/**
 * 定时执行删除过期缓存
 */
@Component
@Slf4j
public class ExpireCacheEvictJob {

    @Autowired
    private ExpireCacheManager cacheManager;
    /**
     * 缓存名与缓存时间
     */
    private static Map<String, Long> cacheNameExpireMap;
    // 可以优化到配置文件或字典中
    static {
        cacheNameExpireMap = new HashMap<>(5);
        cacheNameExpireMap.put("getUserById", 180000L);
        cacheNameExpireMap.put("getUsersByName", 300000L);
    }

    /**
     * 5分钟执行一次
     */
    @Scheduled(fixedRate = 300000)
    public void cacheEvict() {
        Long now = System.currentTimeMillis();
        // 获取所有缓存
        Collection<String> cacheNames = cacheManager.getCacheNames();
        for (String cacheName : cacheNames) {
            // 该类缓存设置的过期时间
            Long expire = cacheNameExpireMap.get(cacheName);
            // 获取该缓存的缓存内容集合
            Cache cache = cacheManager.getCache(cacheName);
            ConcurrentMap<Object, Object> store = (ConcurrentMap) cache.getNativeCache();
            Set<Object> keySet = store.keySet();
            // 循环获取缓存键值对，根据value中存储的插入时间，判断key是否已过期，过期则删除
            keySet.stream().forEach(key -> {
                // 缓存内容包装对象
                ExpireCacheWrap value = (ExpireCacheWrap) store.get(key);
                // 缓存内容插入时间
                Long insertTime = value.getInsertTime();
                if ((insertTime + expire) < now) {
                    cache.evict(key);
                    log.info("key={},insertTime={},expire={},过期删除", key, insertTime, expire);
                }
            });
        }

    }
}

Through the above operations, it is realized that ConcurrentMapCacheManagerthe support expires and is automatically deleted, and the developer is basically unaware and intrusive. You only need to configure the cache time in the configuration file.

But if my project already supports a third-party cache such as Redis, how can I graft this function to Redis based on the principle of not using it in vain?

It just so happens that our project is introducing R2m recently, so let’s try it out^-^.

To be continued~Thanks~

Author: Guo Yanhong, Jingdong Technology

Source: JD Cloud Developer Community