Redis occupies memory size
We know that Redis is a memory-based key-value database. Because the memory size of the system is limited, we can configure the maximum memory size that Redis can use when using Redis.
1. Configure via configuration file
Set the memory size by adding the following configuration to the redis.conf configuration file under the Redis installation directory
//设置Redis最大占用内存大小为100M
maxmemory 100mb
The redis configuration file does not necessarily use the redis.conf file under the installation directory. When starting the redis service, you can pass a parameter to specify the redis configuration file.
2. Modify through commands
Redis supports dynamic modification of memory size through commands at runtime
//设置Redis最大占用内存大小为100M
127.0.0.1:6379> config set maxmemory 100mb
//获取设置的Redis能使用的最大内存大小
127.0.0.1:6379> config get maxmemory
If you do not set the maximum memory size or set the maximum memory size to 0, the memory size is not limited under the 64-bit operating system, and the maximum memory size is 3GB under the 32-bit operating system
Redis memory elimination
Since the maximum memory size of Redis can be set, the configured memory will be used up. Then when the memory is used up, if you continue to add data to Redis, will there be no memory available?
In fact, Redis defines several strategies to deal with this situation:
noeviction (default strategy): No more services are provided for write requests, and an error is returned directly (except for DEL requests and some special requests)
allkeys-lru: Use the LRU algorithm to eliminate all keys
Volatile-lru: Use the LRU algorithm to eliminate keys with an expiration time set
allkeys-random: randomly eliminate data from all keys
volatile-random: Randomly eliminated from the keys with expiration time set
Volatile-ttl: Among the keys with expiration time set, they are eliminated according to the expiration time of the key. The earlier the expiration time, the priority will be eliminated
When using the three strategies of volatile-lru, volatile-random, and volatile-ttl, if no key can be eliminated, it will return an error like noeviction.
How to obtain and set the memory elimination strategy
Get the current memory elimination strategy:
127.0.0.1:6379> config get maxmemory-policy
Set the elimination strategy through the configuration file (modify the redis.conf file):
maxmemory-policy allkeys-lru
Modify the elimination strategy through commands:
127.0.0.1:6379> config set maxmemory-policy allkeys-lru
LRU algorithm
What is LRU?
As mentioned above, the maximum memory that can be used by Redis is exhausted, and the LRU algorithm can be used to eliminate memory. So what is the LRU algorithm?
LRU (Least Recently Used), which is the least recently used, is a cache replacement algorithm. When using memory as a cache, the size of the cache is generally fixed. When the cache is full, at this time continue to add data to the cache, it is necessary to eliminate a part of the old data, free up memory space for storing new data. At this time, the LRU algorithm can be used. The core idea is: if a piece of data has not been used in the recent period, it is unlikely to be used in the future, so it can be eliminated.
Implementation of LRU in Redis
Approximate LRU algorithm
Redis uses the approximate LRU algorithm, which is not the same as the conventional LRU algorithm. The approximate LRU algorithm eliminates data through random sampling, and randomly generates 5 (default) keys each time, and eliminates the least recently used key from it.
The number of samples can be modified through the maxmemory-samples parameter: Example: maxmemory-samples 10 The larger the maxmenory-samples configuration is, the closer the result of elimination is to the strict LRU algorithm
Redis In order to achieve the approximate LRU algorithm, an additional increase is added to each key A 24-bit field is used to store the time when the key was last accessed.
Redis3.0 optimization of approximate LRU
Redis3.0 has made some optimizations to the approximate LRU algorithm. The new algorithm maintains a candidate pool (size 16). The data in the pool is sorted according to the access time. The first randomly selected key will be placed in the pool, and then each randomly selected key will only be accessed when the access time is less than the pool. The minimum time will be placed in the pool until the candidate pool is full. When it is full, if there is a new key that needs to be put in, the last access time (recently accessed) in the pool is removed.
When it needs to be eliminated, the key with the least recent access time (the longest not accessed) is directly selected from the pool and eliminated.
Comparison of LRU algorithm
We can compare the accuracy of each LRU algorithm through an experiment. First, add a certain amount of data n to Redis to use up the available memory of Redis, and then add n/2 new data to Redis. At this time, we need to eliminate some of them. If you follow the strict LRU algorithm, the n/2 data that should be added first should be eliminated. Generate the following comparison chart of each LRU algorithm (picture source):
You can see that there are three different colored points in the picture:
Light gray is the data
that has been eliminated. Gray is the old data that has not been eliminated.
Green is the newly added data.
We can see that the redis3.0 sample number is 10 and the graph generated is the closest to the strict LRU. With the same use of 5 samples, Redis3.0 is better than Redis2.8.
LFU algorithm
The LFU algorithm is a new elimination strategy added in Redis 4.0. Its full name is Least Frequently Used, and its core idea is to eliminate the key according to the frequency of its recent visits. The seldom-visited ones will be eliminated first, and the most visited ones will be kept.
The LFU algorithm can better indicate how hot a key is being accessed. If you are using the LRU algorithm, a key has not been accessed for a long time, but only occasionally accessed once, then it is considered hot data and will not be eliminated, and some keys are likely to be accessed in the future Was eliminated. This will not happen if the LFU algorithm is used, because using it once does not make a key a hot data.
There are two strategies for LFU:
Volatile-lfu: Use the LFU algorithm to eliminate keys in the keys with the expiration time set
allkeys-lfu: Use the LFU algorithm to eliminate data in all keys. The
use of these two elimination strategies is the same as the previous ones, but one thing to note is The two-week strategy can only be set in Redis4.0 and above, if it is set below Redis4.0, an error will be reported
problem
Finally, there is a small question. Some people may have noticed. I did not explain why Redis uses the approximate LRU algorithm instead of the accurate LRU algorithm. You can give your answer in the comment area and discuss and study together.
