This article originated from the translation invitation of Concurrent Programming Network. It translated the content about caching technology in Jakob Jenkov's "Software Architecture" . Although it is a 2014 article, it is not outdated from the software architecture level.
Cache
Caching is a technology that speeds up data search (data reading). It directly reads locally cached data instead of reading data from data sources. Data sources include databases and other remote systems.
The cache is a piece of storage space that is closer to the user than the source data, and can be read faster. The storage medium of the cache is generally memory or disk. In many cases, memory is selected as the cache medium, but the memory cache will lose data when the system restarts.
In a software system, data cache has a multi-layer cache level or a multi-layer cache system. In a web application, there are at least three storage locations for the cache, as shown in the following figure:
In web applications, we use a variety of databases to store data. These databases can store data in memory so that we can read them directly without reading data from disk. The web server can cache pictures, css files, js files, etc. in memory, without accessing files from the hard disk every time it is needed. The web application can cache the data read from the database, so there is no need to read data from the database through the network every time it is used. Finally, the browser may also store static files and data. In browsers that support HTML5, there are technical support caching such as localstorage storage space, application data caching, and local sql storage.
When we talk about caching, there are several things to consider:
- Write cache
- Keep cache and remote system data in sync
- Manage cache size
I will discuss these items in the following content.
Write cache
The first challenge is to read data from the remote system and write it to the cache. There are generally two ways:
- Write ahead cache
- Write cache
The write-ahead cache is to cache the required data when the system is started. To do this, you need to know in advance which data needs to be cached. But sometimes we don't know which data needs to be cached when the system is started.
Write-time cache means that when the data is used for the first time, the data is cached, and then the data in the cache can be used. The method of this operation is to first check whether there is data in the cache, and use it directly, if there is not, read the data from the remote system and then write it to the cache.
In the following table, I have listed the advantages and disadvantages of writing in advance and writing in time:
| advantage | Disadvantage | |
|---|---|---|
| Write ahead cache | Reduce the delay of the first cache data compared to write time | When the system starts to initialize the cached data, it takes a relatively long time. Moreover, it is possible that the cached data will never be used. |
| Write cache | The cached data is the data that needs to be used, and there is no startup delay | When data is cached for the first time, it takes a long time, which may lead to inconsistent user experience |
Of course, in actual practice, we may use two methods together: we can use the pre-caching method for hot data, and the time-consuming caching method for other data.
Keep cache and remote system data in sync
A huge challenge of caching data is to keep the cached data synchronized with the remote system data, that is, the data is consistent. Depending on the system structure, there are generally different ways to achieve this. Let's talk about these ways.
Direct caching
Write-through caching is a way to allow read-write caching. In this way, the computer that saves the cached data writes the data to the remote system while writing the data to the cache. Simply put, the write operation is written to the remote system.
This method works only when the data of the remote system can only be modified by the write-through cache. If all data reads and writes have to go through a write-through cache system, it is easy to update the written data to the remote system to maintain the consistency of the cache and the remote system data.
Based on expiration time
If the remote system can update data without relying on the remote system, data synchronization between the cache and the remote system will be difficult to ensure through write-through caching.
One way to keep cached data in sync is to set a cache time for the data. When the data expires, the data is cleared from the cache. If you need to read the data again, you can read the latest data from the remote system and cache it.
The data expiration time depends on the needs of the system. Some types of data (such as articles) may not need to be completely updated at any time. An expiration time of 1 hour can be set. For some articles, you can even tolerate an expiration time of 24 hours.
It should be noted that if the expiration time is relatively short, the remote system may be read frequently, reducing the effect of the cache.
Active expiration
Another way is to actively expire, which means to actively update the cached data. For example, when the remote system data is updated, a message is sent to the cache system to indicate that the system data has been updated, and the data can be set to expire.
The advantage of active expiration is that it may ensure that the cached data is updated as soon as possible after the remote system data is updated. There is also an additional benefit that the "based on expiration time" method cannot be achieved, that is, the unmodified data will not be updated frequently.
The disadvantage of active expiration is that it needs to be able to detect changes in remote system data. If the remote system is a relational database and data can be updated by different mechanisms, each update mechanism needs to report which data they have updated. Otherwise, there is no way to notify the system that caches the data of expiration messages.
Manage cache size
Managing cache size is an important aspect. Many systems store so much data that it is impossible to store all the data in the cache. Therefore, a mechanism is needed to manage the amount of cached data. Managing the cache size is usually to clear unnecessary cache data to free up enough space. There are generally the following ways:
- Time-based cleanup
- First in first out (FIFO)
- First in Last Out (FILO)
- Least used
- Minimum visit interval
The time-based cleaning method is similar to the time-based expiration mentioned earlier. In addition to keeping the data synchronized with the remote system, it can also reduce the size of cached data. You can start a separate monitor thread, or you can clean up data when reading and writing new values.
The first-in first-out cleaning method means that when a new cache is written, the earliest inserted cache value needs to be deleted. If there is enough space, it is not necessary to delete any data.
The first-in-last-out method is just the opposite of the first-in-first-out method. This method is more useful for the case of hot data when the data is stored first.
The least used cleaning method is to clean the cached data with the least number of accesses first. The purpose of this method is to avoid cleaning up hot data. In order to achieve this method, it is necessary to record the number of times the cached data is accessed. There is a problem that needs to be noted. The old values in the cache may have a high number of accesses, which means that these old values will not be cleaned up. For example, the cache of an old article has been accessed many times before, but it has been rarely accessed recently, but because the original access volume is high, although the current access volume is low, it will not be cleared. To avoid this situation, the number of visits can be counted for N hours.
The minimum visit interval cleaning method is to take the visit interval into account. When accessing a certain cached data, it is necessary to mark the time of accessing the data and increase the number of accesses. When accessing this cached data for the second time, increase the number of accesses and calculate the average access time. Those data that used to be hotspots were frequently accessed, but the recently accessed time interval has become longer and the access frequency has decreased, the average access time will be reduced, and when it drops to a sufficiently low level, it will be cleaned up.
One way to change this is to only calculate the time of the last N visits. N can be 100, 1, or any other meaningful number. Whenever the access count reaches N, the access count is reset to 0, and the access time is recorded. In this way, the data that has fallen in heat can be cleaned up faster.
Another variation is to reset the visit count regularly and only use the cleanup method with minimal visits. For example, for every hour of data cached, the access count of the previous hour will be stored in another variable for use in decision-making cleanup. The visit count for the next hour is reset to 0. This mechanism has the same effect as the last change.
The difference between the last two variants can be summed up as whether the access count has reached N or the time interval has exceeded Y each time the cache is checked. The first way is to access the system clock every N times, and the second way is to read the system clock every time it is accessed (check whether the time interval has expired). Because checking an integer is usually faster than reading the system clock, I will choose the first method.
Remember, even if you use a cache size management system, you need to clean, read, and store data to ensure that they can be consistent with the remote system. Although the cached data is accessed in large numbers and resides in the system, sometimes it needs to be synchronized with the remote system.
Cache in server cluster
The cache design in a single service is simpler, because you can guarantee that all write operations go through a server, and write-through caching can be used. But in a distributed cluster, the situation will be more complicated. The following figure illustrates this situation:
Simply using the write-through cache will only update the cache on the write-operated server, and other servers in the cluster are completely unaware of this and will not update the data.
In a server cluster, a time-based expiration strategy or an active expiration strategy can be used to ensure the synchronization of cached data with remote systems.
Cached products
It is not difficult to implement your own caching system, depending on whether you need deep customization. If it is not necessary to implement the caching system yourself, you can use existing caching products. such as:
I don't know whether these products can meet the needs, but I know they are widely used.
Personal homepage: https://www.howardliu.cn
personal blog post: software architecture-caching technology
CSDN homepage: http://blog.csdn.net/liuxinghao
CSDN blog post: software architecture-caching technology
