Guava Cache is an excellent local caching framework that provides a simple and easy-to-use API for developers to use.
In this article, we talk about how to use Guava Cache asynchronous refresh techniques to improve system performance.
1 Basic usage
First, add maven dependencies in the Java application:
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>31.0.1-jre</version>
</dependency>
Then write the test case:
LoadingCache is a local caching tool that supports configuring loading functions , scheduled expiration and other functions.
In the example, a capacity-based recycling policy is configured , the maximum cache capacity is set to 100, and scheduled invalidation and refresh functions are configured .
- Timing failure
When configured expireAfterWrite, cache items expire within a specified amount of time after they are created or last updated.
- Refresh function
Configuration refreshAfterWriteSets a refresh time so that new values can be reloaded when cached items expire.
We simulate the process of executing the load method after expiration/reloading and executing the reload method. The execution results are shown in the figure below:
The execution results show that: Guava Cache does not have a background task thread to asynchronously execute the load or reload method.
-
expireAfterWriteAllow one thread to execute the load method, while other threads block and wait.When a large number of threads obtain cached values with the same key, only one thread will enter the load method, while other threads wait until the cached value is generated. This also avoids the risk of cache breakdown. In high concurrency scenarios, this will still block a large number of threads.
-
refreshAfterWriteAllow one thread to execute the load method and other threads to return the old value.Under single key concurrency, using refreshAfterWrite will not block, but if multiple keys happen to expire at the same time, it will still put pressure on the database.
In order to improve system performance, we can deal with it from the following two aspects:
- To reduce the frequency of expiration, that is, to reduce the frequency of executing the load method, configure refresh < expire.
- Adopt an asynchronous refresh strategy, that is, the thread loads data asynchronously, during which all requests return the old cached value .
2 Two ways to implement asynchronous refresh
2.1 Override the reload method
2.2 Implement asyncReloading method
No matter which solution is used, a separate thread pool needs to be defined to perform refresh tasks.
3 Asynchronous refresh + multi-level cache
In 2018, an e-commerce company that the author worked for needed to optimize the performance of the app homepage interface. It took the author about two days to complete the entire solution, using a two-level cache mode and Guava's asynchronous refresh mechanism.
The overall architecture is shown in the figure below:
The cache reading process is as follows:
1. When the business gateway is just started, there is no data in the local cache. Read the Redis cache. If there is no data in the Redis cache, call the shopping guide service through RPC to read the data, and then write the data to the local cache and Redis; if the Redis cache If not empty, the cached data will be written to the local cache.
2. Since the local cache has been warmed up in step 1, subsequent requests directly read the local cache and return it to the user.
3. Guava is configured with a refresh mechanism, which will call the custom LoadingCache thread pool (5 maximum threads, 5 core threads) every once in a while to synchronize data from the shopping guide service to the local cache and Redis.
After optimization, the performance is very good, the average time consumption is about 5ms, and the frequency of applying GC is greatly reduced.
This solution still has flaws. One night we found that the data displayed on the home page of the app was sometimes the same and sometimes different.
That is to say: Although the LoadingCache thread has been calling the interface to update the cache information, the data in the local cache of each server is not completely consistent.
This illustrates two very important points:
1. Lazy loading may still cause data inconsistency on multiple machines;
2. The number of LoadingCache thread pools is not configured reasonably, resulting in a pile-up of tasks.
Ultimately, our solution was:
1. Asynchronous refresh combines the message mechanism to update the cache data, that is: when the configuration of the shopping guide service changes, the business gateway is notified to re-pull the data and update the cache.
2. Appropriately increase the thread pool parameters of LoadingCache, and bury points in the thread pool to monitor the usage of the thread pool. When the thread is busy, an alarm can be issued, and then the thread pool parameters can be modified dynamically.
4 Summary
Guava Cache is very powerful, but it does not have a background task thread to asynchronously execute the load or reload method. Instead, it performs related operations through request threads.
In order to improve system performance, we can deal with it from the following two aspects:
- To reduce the frequency of expiration, that is, to reduce the frequency of executing the load method, configure refresh < expire.
- Adopt an asynchronous refresh strategy, that is, the thread loads data asynchronously, during which all requests return the old cached value .
The author once optimized the homepage interface of an e-commerce website. The solution used was: Guava's asynchronous refresh mechanism + multi-level cache, which achieved very good optimization results.
Of course, when we use this method, we still need to consider the consistency of the data.
References:
If my article is helpful to you, please like it, read it, and forward it . Your support will encourage me to produce higher-quality articles. Thank you very much!
