[Abstract] This article introduces the principle of Redis caching, analyzes the caching model, cache consistency and cache exception scenarios in detail.
[Author] Li Jie focuses on the exploration and research of Java virtual machine technology and cloud native technology.
Although (relational) database systems (SQL) bring many excellent properties, such as ACID, in order to maintain these properties, the performance of the database often appears to be stretched and pale in the "3 high" condition environment.
In order to solve this problem, we often need to add a caching layer between the application layer (that is, the back-end code that handles business logic) and the storage layer (that is, the SQL database). This cache layer is usually implemented using memory cache. After all, the performance bottleneck of traditional SQL databases usually occurs at the I/O level of secondary storage (ie, hard disk). As the price of main memory (RAM) has dropped over the past decade, storing (at least part of) the data in main memory to improve performance is a cost-effective solution. Based on the current technological development status, Redis has become a more popular choice today.
Of course, most systems only store so-called "hot data" in the cache layer (i.e., main memory). Based on the Pareto principle (also known as the 80/20 rule), for most events, approximately 80% of the effects come from 20% of the causes. To save costs, we only need to store these 20% in the cache layer. To identify "hot data", we can specify an eviction policy (such as LFU or LRU) to determine which data will expire.
Caching overview
Caching is a "warm-up" technique used to store frequently accessed data in temporary storage (called cache) to reduce hard drive read/writes. Caching is everywhere, and the performance of Web applications can be greatly improved based on this technology.
Typically, in the original monolithic architecture model, when a user sends a message request to our service, the web server will first read or write to the database and then return the response. In the case of caching, the server first checks if the cached copy exists and if so returns the data from the cache instead of asking the database. It saves time and database computational effort.
The following is a brief introduction to how the application requests Redis. This is mainly based on the cluster of Master-Slave-Sentinel mode. The App communicates with Redis Sentinel by calling Redis Client, such as Jedis, Lettuce and Redisson. When Redis Master switches to Slave , the Application can still work normally, as shown below is the detailed timing diagram:
cache model
In distributed systems, based on the guidance of the CAP theorem, these strategies are selected according to business needs and context, and can usually be divided into regular mode and Cache-Aside mode. Before we begin, let us understand the common caching modes by refreshing the cache, as follows:
Write model
1. Write Through: that is, "direct writing". This model is cached after being written to the database synchronously. This is safe because it writes to the database first, but is slower than writing later. It provides better performance for write-before-read scenarios compared to write invalidation. In this write strategy, data is written to the cache first and then to the database. The cache is inline with the database, and writes always go through the cache to the primary database.
The algorithm for direct writing mode is:
1) For immutable operations (read):
This strategy does not handle immutable operations. It should be combined with read-through mode.
2) For variable operations (create, update, delete):
The client only needs to create, update or delete entries in Redis. The cache layer must synchronize this change to MySQL atomically.
The shortcomings of direct writing mode are also obvious. First, many caching layers don't support this natively. Secondly, Redis is a cache and not an RDBMS. It is not designed to be flexible. Therefore, changes may be lost before being replicated to MySQL. Even though Redis now supports persistence technologies such as RDB and AOF, this approach is still not recommended.
On their own, write-through caches don't seem to do much, and in fact, they introduce additional write latency because data is written to the cache first and then to the main database. But when paired with a read-through cache, we get all the benefits of a read-through, and we also get data consistency guarantees that save us from using cache invalidation techniques.
DynamoDB Accelerator (DAX) is a good example of a read/write cache. It works inline with DynamoDB and applications. DynamoDB can be read and written through DAX.
2. Write Behind: that is, "write behind or write back". Based on this strategy, the application writes data to the cache, which immediately acknowledges it and writes the data back to the database after a delay. This is very fast for writes, even faster if multiple writes on the same key are combined into a single write to the database. But the database is inconsistent with the cache for a long time, and if the process crashes before the data is flushed to the database, data may be lost. RAID cards are a good example of this model. To avoid data loss, a battery backup unit on the RAID card is often required to keep the data in cache but not yet logged to disk.
The algorithm for Write Behind mode is:
1) For immutable operations (read):
This strategy does not handle immutable operations. It should be combined with read-through mode.
2) For variable operations (create, update, delete):
The client only needs to create, update or delete entries in Redis. The cache layer saves the changes to the message queue and returns success to the client. Changes are replicated asynchronously to MySQL and may occur after Redis sends a success response to the client.
Write-behind mode differs from write-through in that it copies changes to MySQL asynchronously. It improves throughput because clients don't have to wait for replication to happen. A message queue with high durability might be a possible implementation. Redis Streaming (supported since Redis 5.0) might be a good choice. To further improve performance, changes can be combined and updated in batches in MySQL (to save query times).
The disadvantages of Write Behind mode are similar. First, many caching layers don't support this natively. Secondly, the message queue used must be FIFO (first in, first out). Otherwise, updates to MySQL may be out of order, so the end result may be incorrect.
Write-back caching improves write performance and is suitable for write-heavy workloads. When used in conjunction with read-through, it is suitable for mixed workloads where recently updated and accessed data is always available in cache.
It is resilient to database failures and can tolerate some database downtime. If batching or merging is supported, it can reduce overall writes to the database, thereby reducing load and lowering costs, if the database provider charges by the number of requests, such as Dynamic Database. Keep in mind that DAX is write-through, so if your application is write-heavy, you won't see any cost reduction.
Some developers use Redis for caching and writeback to better absorb spikes during peak load. The main disadvantage is that if the cache fails, the data may be permanently lost.
Most relational database storage engines (i.e. InnoDB) have write-back caching enabled by default within them. Queries are first written to memory and eventually flushed to disk.
3. Write invalidate: Similar to direct write, first write to the database and then invalidate the cache. This simplifies handling of consistency between cache and database in the case of concurrent updates. We don't need complex synchronization, the trade-off is a lower hit rate since we always invalidate the cache and the next read will always miss.
Read model
Read Through: That is, "read through". When a read misses, it needs to be loaded from the database and saved to the cache. The main problem with this model is that the cache sometimes needs to be warmed up based on certain scenarios. The read-through cache remains consistent with the database. When a cache miss occurs, it loads the missing data from the database, fills the cache and returns it to the application.
The algorithm for read-through mode is:
1. For immutable operations (read):
The client will always simply read from the cache. Cache hits or cache misses are transparent to the client. If it is a cache miss, the cache should have the ability to automatically retrieve it from the database.
2. For variable operations (create, update, delete):
This policy does not handle mutable operations. It should be used in conjunction with write-through (or write-behind) mode.
A major disadvantage of read-through mode is that many cache layers may not support it. For example, Redis will not be automatically obtained from MySQL (unless a plug-in is written for Redis).
Cache-Aside and Read-Through strategies both lazily load data, that is, it is only loaded on the first read. Its applicable use case scenarios are as follows:
Although Read-Through and Cache-Aside are very similar, there are at least two key differences:
On the cache side, the application is responsible for fetching data from the database and filling the cache. In read-through, this logic is typically supported by a library or standalone cache provider.
Unlike Cache-Aside, the data model in Read-Through Cache cannot be different from the data model of the database.
Read-through caching is best suited for read-heavy workloads when the same data is requested multiple times. For example, a news story. The disadvantage is that it always results in a cache miss when the data is requested for the first time and incurs the additional penalty of loading the data into the cache. Developers deal with this problem by manually issuing queries to "heat" or "warm up" the cache. Just like cache-aside, it is possible for data to be inconsistent between cache and database, and the solution lies in the write strategy, as we will see below.
Do not read or write model
Refresh ahead: Predict hot data and automatically refresh the cache in the database, never blocking reads, best for small read-only data sets, such as zip code list cache, we can refresh the entire cache periodically because it is small and read-only . If you can accurately predict which keys will be read most often, you can also warm them up in this mode. Finally, this mode may have to be used if data is updated outside the system and the system cannot be notified.
In most scenarios, we usually use models such as read-through and write-through/write-behind/write-invalid. For the Refresh-ahead model, it can be used alone or as an optimization to predict and warm-up reads for read-through. There are two implementation modes for who is responsible for cache maintenance, the caller or the dedicated layer.
1. Cache-Facade: The cache layer is a library or service that delegates writing to the database. We only talk to the cache layer. The database is then transparent to our application. The caching layer handles consistency and failover. For example, many databases have their own cache, which is a good example of what caching can look like. We can also write some in-process DAO layer to read/write entities with an embedded caching layer. From the caller's perspective, this small layer is also a caching facade.
2. Cache-Aside: Our application remains cache-coherent, which means the application code is more complex, but this provides greater flexibility. For example, cache-facade patterns like Database Query Cache can only cache rows, if you want to cache a Java POJO or Kotlin data class with rows, it's much easier to leave the cache aside. But it can still use the cache facade, for example, use Spring cache as a facade library to cache POJOs and automatically process POJOs in the database in the background.
We use this cache-side mode when the cache does not support native read-through and write-through operations, and resource requirements are unpredictable.
1) Read: Try to hit cache. If there is no hit, read from the database and then update the cache.
2) Write: First write to the database, then delete the cache entry. A common pitfall here is that people mistakenly update the cache with values, and double writes in high-concurrency environments can make the cache dirty.
In this mode, it is still possible to have a dirty cache. The above happens when these two conditions are met: read the database and update the cache, update the database and delete the cache.
cache consistency
Cache consistency model (reference) diagram
How to ensure data consistency between cache (Redis) and data storage (database), there are usually multiple design and implementation strategies. This article focuses on a brief analysis of the Cache Aside Pattern. This model is also used in actual business It is widely used in scenes. details as follows.
In the Cache Aside Pattern model, the basic process of a write request scenario is usually as follows: first update the DB, and then directly delete the Cache.
In the implementation of business scenarios, if the database is updated successfully but the cache deletion operation fails, simply put, there are usually two main solutions:
1. Shorten the Cache expiration time: We shorten the expiration time of cached data, so that the cache will load data from the database. In addition, this solution is not applicable to scenarios where the cache is operated first and then the database is operated. This solution is usually not recommended in actual business scenarios, and essentially treats the symptoms rather than the root cause.
2. Add a Cache update retry mechanism: If the Cache service is currently unavailable and cache deletion fails, we will retry after a period of time. The number of retries can be set by yourself. If it still fails after multiple retries, we can store the currently updated failed Key in the queue, and then delete the corresponding Key in the cache after the cache service is available. Consider using message queues. This solution is a commonly used solution strategy that can meet the needs of most business scenarios.
In fact, in essence, the planning and design of caching solutions often rely on actual business scenario requirements. After all, technology serves the business. Maybe sometimes after we introduce caching, in order to solve the short-term inconsistency problem, we choose to make the system design more complicated, which is completely unnecessary.
Cache exception scenarios
Caching scene model graph
actually. In actual scenarios, considering various application exceptions and business failures, it is usually impossible to completely use distributed cache and database systems to implement the linear consistency model. Every caching mode has its own limitations and there are cases where we cannot get sequential consistency or sometimes we get unexpected delays between the cache and the database. For all the solutions shown by the author in this article, extreme situations of high concurrency will always be encountered depending on different business needs. So there is no magic bullet for this, understand the limitations and define specific consistency requirements before choosing a solution.
If you want to achieve linear consistency and fault tolerance, it is recommended not to use a caching strategy and consider other solutions. The above is the analysis related to the Redis cache system. I hope it will be useful to everyone.
Activity time~
