07 Redis cache expiration elimination strategy

common interview questions

• What is your redis memory setting in production?
• How to configure and modify the memory size of redis
• what do you do if the memory is full
• How does redis clean up memory? Do you understand regular deletion and lazy deletion?
• redis cache elimination strategy
• Have you ever understood the LRU of redis?

What should I do if the Redis memory is full?

What is the default memory of redis? Where to check? How to set and modify?

• View the maximum memory usage of Redis
• 
• How much redis default memory can be used?
  • If you do not set the maximum memory size or set the maximum memory size to 0, the memory size is not limited under the 64-bit operating system, and the maximum memory size is 3GB under the 32-bit operating system
• How do you configure it in general production? It is generally recommended that Redis set the memory to three quarters of the maximum physical memory
• How to modify redis memory settings
  • By modifying the file configuration
  • 
  • Modify by command
  • 
• What command to view redis memory usage?
  • info memory

What happens if Redis memory usage exceeds the set maximum?

Change the configuration, deliberately set the maximum value to 1 byte to try

• in conclusion
  • The maxmemory option is set, if the redis memory usage reaches the upper limit
  • If the expiration time is not added, the data will be filled with maxmemory. In order to avoid similar situations, the memory elimination strategy in the next chapter is introduced.

Why is the data written in redis gone? How is it deleted?

Deletion strategy for redis expired keys

• If a key is expired, will it be deleted from memory immediately after the expiration time? ?
• If the answer is yes, delete it immediately. Do you go by yourself or the interviewer sends you?
• If not, when will it be deleted after the expiration? ? What is the operation?

Three different deletion strategies

delete immediately

• It is impossible for Redis to traverse all the keys with the lifetime set at all times to detect whether the data has reached the expiration time, and then delete it.
• Immediate deletion can ensure the maximum freshness of data in memory, because it guarantees that expired key values ​​will be deleted immediately after expiration, and the memory it occupies will be released accordingly. But immediate removal is the most cpu-unfriendly. Because the delete operation will take up CPU time, if it happens to be very busy when the CPU is doing calculations such as intersection or sorting, it will cause extra pressure on the CPU, making the CPU tired, and it needs to be deleted from time to time. die. . . . . . .
• This will generate a lot of performance consumption, and it will also affect the data read operation .
• Summary: Unfriendly to CPU, use processor performance for storage space (trade time for space)

lazy delete

• The data reaches the expiration time and is not processed. When the data is accessed next time, if it has not expired, return the data; if it is found to be expired, delete it, and return does not exist.
• The disadvantage of the lazy deletion strategy is that it is the least friendly to memory .
• If a key has expired, and this key is still retained in redis, as long as the expired key is not deleted, the memory it occupies will not be released.
• When using the lazy deletion strategy, if there are a lot of expired keys in the database, and these expired keys happen not to be accessed, then they may never be deleted (unless the user manually executes FLUSHDB), we can even set This situation is regarded as a memory leak – useless garbage data takes up a lot of memory, but the server will not release them by itself, which is definitely not good news for Redis servers whose running status is very dependent on memory
• Summary: Unfriendly to memory, use storage space for processor performance (trade space for time)

regularly delete

• The periodic deletion strategy is a compromise between the previous two strategies: the periodic deletion strategy executes the operation of deleting expired keys at regular intervals , and reduces the impact of deletion operations on CPU time by limiting the duration and frequency of deletion operations.

• Periodically poll the time-sensitive data in the redis library, adopt a random extraction strategy, and use the proportion of expired data to control the deletion frequency

  • Feature 1: The CPU performance usage setting has a peak value, and the detection frequency can be customized
  • Feature 2: The memory pressure is not very high, and the cold data that occupies the memory for a long time will be continuously cleaned up

• Summary: periodic spot checks on storage space (random spot checks, key spot checks)

• By default, redis checks every 100ms to see if there is an expired key, and if there is an expired key, it will be deleted. Note: redis does not check all the keys every 100ms, but randomly selects them for checking (if every 100ms, all keys are checked, redis goes directly to the ICU). Therefore, if only the regular deletion strategy is adopted, many keys will not be deleted by the time.

• The difficulty of the periodic deletion strategy is to determine the duration and frequency of the deletion operation: if the deletion operation is performed too frequently, or the execution time is too long, the periodic deletion strategy will degenerate into an immediate deletion strategy, so that the CPU time is consumed too much Above the delete expired key. If the deletion operation is performed too little, or the execution time is too short, the regular deletion strategy will be the same as the lazy deletion strategy, and memory will be wasted. Therefore, if a periodic deletion strategy is adopted, the server must reasonably set the execution time and frequency of the deletion operation according to the situation .

• The above steps have been passed, are there any loopholes?

  • When regularly deleted, it was never spot-checked
  • When lazy deleting, it has never been used in the point
  • The above 2 steps ======> A large number of expired keys are accumulated in the memory, causing the redis memory space to be tight or exhausted soon
  • There must be a better back-and-forth solution...

redis cache elimination strategy

What are there (redis6.0.8 version)

• noeviction: will not evict any key
• allkeys-lru: delete all keys using the LRU algorithm
• volatile-lru: Use the LRU algorithm to delete all keys with an expiration time set
• allkeys-random: Randomly delete all keys
• volatile-random: Randomly delete all keys with an expiration time set
• volatile-ttl: Delete keys that are about to expire
• allkeys-lfu: use LFU algorithm to delete all keys
• volatile-lfu: Use the LFU algorithm to delete all keys with an expiration time set