Redi sets the expiration key and Redis's expiration key deletion strategy

Redis set expiration key

Through EXPIRE command or PEXPIRE command, the client can set the time to live (Time To Live, TTL) for a key in the database in seconds or milliseconds. After the specified number of seconds or milliseconds, the server will automatically delete the survival Key with time 0:

Similar to the EXPIRE command and PEXPIRE command, the client can use the EXPIREAT command or the PEXPIREAT command to set the expiration time (expire time) for a key in the database in seconds or milliseconds. The expiration time is a UNIX timestamp. When the key expires When time comes, the server will automatically delete this key from the database:

The TTL command and the PTTL command accept a key with a time to live or expiration time, and return the remaining time to live of the key, that is, how long it is until the key is automatically deleted by the server:

How does Redis set the expiration time?

Redis has four different commands that can be used to set the key lifetime (how long the key can exist) or expiration time (when the key will be deleted):

• EXPIRE<key> <ttl>命令用于将键key的生存时间设置为ttl秒。
• PEXPIRE<key> <ttl>命令用于将键key的生存时间设置为ttl毫秒。
  -EXPIREAT<key> <timestamp>命令用于将键key的过期时间设置为timestamp所指定的秒数时间戳。
• PEXPIREAT<key> <timestamp>命令用于将键key的过期时间设置为timestamp所指定的毫秒数时间戳。

Although there are many different units and different forms of setting commands, in fact, the three commands EXPIRE, PEXPIRE, and EXPIREAT are all realized by using the PEXPIREAT command: no matter which of the above four commands is executed by the client, it will be converted. After that, the final execution effect is the same as executing the PEXPIREAT command.

How does Redis save the expiration time?

The expires dictionary of the redisDb structure stores the expiration time of all keys in the database. We call this dictionary the expired dictionary:

• The key of the expired dictionary is a pointer that points to a key object in the key space (that is, a database key).
• The value of the expiration dictionary is an integer of type long long. This integer holds the expiration time of the database key pointed to by the key --- a UNIX timestamp with millisecond precision.

The following figure shows an example of a database with an expired dictionary. In this example, the key space stores all key-value pairs in the database, and the expired dictionary stores the expiration time of the database key.

The expired dictionary in the figure saves two key-value pairs:

• The key of the first key-value pair is an alphabet key object with a value of 1385877600000, which means that the expiration time of the database key alphabet is 1385877600000 (at 0:00 on December 1, 2013).
• The key of the second key-value pair is the book key object, with a value of 1388556000000, which means that the expiration time of the database key book is 138855600000o0 (at 0:00 on January 1, 2014).

other

How does Redis remove the expiration time?

The PERSIST command is the reverse operation of the PEXPIREAT command: the PERSIST command looks up the given key in the expired dictionary and disassociates the key and value (expiration time) in the expired dictionary.

How does Redis calculate and return the remaining survival time?

How does Redis determine whether a key has expired?

With the expiration dictionary, the program can use the following steps to check whether a given key has expired:

• Check whether the given key exists in the expiration dictionary: if it exists, get the expiration time of the key.
• Check whether the current UNIX timestamp is greater than the expiration time of the key: if it is, then the key has expired; otherwise, the key has not expired.

Expired key deletion strategy

After the above introduction, we know that the expiration time of the database key is stored in the expiration dictionary, and we know how to judge whether a key expires based on the expiration time. Now the remaining question is: if a key expires, then what is it Will it be deleted?

Three different deletion strategies:

• Timed deletion: While setting the key expiration time, create a timer (timer), let the timer execute the key deletion operation immediately when the key expiration time comes.
• Lazy deletion: Let the key expire regardless, but every time you get a key from the key space, check whether the obtained key is expired, if it expires, delete the key; if it does not expire, return the key.
• Periodic deletion: every certain period of time, the program will check the database and delete the expired keys. As for how many expired keys to delete and how many databases to check, it is up to the algorithm.
  Among these three strategies, the first and third are active deletion strategies, and the second is passive deletion strategies.

Among these three strategies, the first and third are active deletion strategies, and the second is passive deletion strategies.

Timed delete

Timed deletion strategy** is the most memory-friendly: **By using a timer, the timed deletion strategy can ensure that expired keys will be deleted as quickly as possible, and release the memory occupied by expired keys.

On the other hand, regularly delete strategies disadvantage is that it is the most unfriendly of CPU time is: under the relatively large number of expired keys, the delete key expired this behavior may consume a considerable amount of CPU time , memory was not nervous but the CPU When time is very tight, using CPU time to delete expired keys that have nothing to do with the current task will undoubtedly affect the server's response time and throughput.

Lazy deletion

The lazy deletion strategy is the most friendly to CPU time : the program will only check the expiration of the key when the key is taken out, which can ensure that the operation of deleting the expired key will only be carried out in the case of necessity, and the target of deletion Limited to the currently processed keys, this strategy will not spend any CPU time on deleting other irrelevant expired keys.

The drawback is inert deletion policy, which memory is the most unfriendly : If a key has expired, and this bond has still retained in the database, as long as the key is not expired is deleted, it will not release the memory occupied , It may cause a memory leak .

Implementation of lazy deletion

See redis design and implementation 9.6.1

Delete regularly

The regular deletion strategy is an integration and compromise of the first two strategies:

• The regular deletion strategy executes the delete expiration key operation at regular intervals, and reduces the impact of the delete operation on the CPU time by limiting the duration and frequency of the delete operation.
• In addition, by periodically deleting expired keys, the periodic deletion strategy effectively reduces the memory waste caused by expired keys.

The difficulty of a regular deletion strategy is to determine the duration and frequency of the deletion operation:

• If the delete operation is performed too frequently, or the execution time is too long, the regular deletion strategy will degenerate into a regular deletion strategy, so that too much CPU time is consumed on deleting the expired key.
• If the deletion operation is performed too little or the execution time is too short, the regular deletion strategy will be the same as the lazy deletion strategy, which will cause a waste of memory.

Therefore, if a regular deletion strategy is adopted, the server must reasonably set the execution duration and execution frequency of the deletion operation according to the situation.

Implementations that are periodically deleted

See redis design and implementation 9.6.2

Processing of expired keys by AOF, RDB and copy functions

AOF's handling of expired keys

1. AOF file writing

   When the server is running in AOF persistence mode, if a key in the database has expired, but it has not been lazily deleted or deleted regularly, then the AOF file will not have any impact due to the expired key.

   When the expired key is deleted lazily or periodically, the program will append (( append) a DEL command to the AOF file to explicitly record that the key has been deleted.

   For example, if the client uses the GET message command to try to access the expired message key, the server will perform the following three actions:

   • Delete the message key from the database.
   • Append a DEL message command to the AOF file.
   • Return an empty reply to the client executing the GET command.

2. AOF file loading

   Similar to the generation of RDB files, in the process of performing AOF rewriting, the program will check the keys in the database, and the expired keys will not be saved in the rewritten AOF file.

   For example, if the database contains three keys k1, k2, k3, and k2 has expired, then during the rewriting work, the program will only rewrite k1 and k3, and k2 will be ignored.

   Therefore, the inclusion of expired keys in the database will not affect the AOF rewrite.

RDB's handling of expired keys

1. Generate RDB file

   When executing the SAVE command or the BGSAVE command to create a new RDB file, the program will check the keys in the database, and the expired keys will not be saved in the newly created RDB file .

   For example, if the database contains three keys k1, k2, k3, and k2 has expired, then when the SAVE command or BGSAVE command is executed, the program will only save the data of k1 and k3 to the RDB file, while k2 is Will be ignored.

   Therefore, the inclusion of expired keys in the database will not affect the generation of new RDB files.

2. Load RDB file

   When starting the Redis server, if the server has the RDB function enabled, the server will load the RDB file:

   • If the server is running in the master server mode , when loading the RDB file, the program will check the keys saved in the file, **unexpired keys will be loaded into the database, and expired keys will be ignored,* *So the expired key will not affect the main server that loads the RDB file.

   • If the server runs in the slave server mode , when loading the RDB file, all the keys stored in the file, regardless of whether they expire or not, will be loaded into the database. However, because the database of the slave server will be emptied when the master-slave server is synchronizing data, generally speaking, the expired key will not affect the slave server loading the RDB file.

   For example, if the database contains three keys k1, k2, k3, and k2 has expired, then when the server starts:

   • If the server is running in master server mode, the program will only load k1 and k3 into the database, and k2 will be ignored.
   • If the server is running in slave server mode, then k1, k2, and k3 will all be loaded into the database.

Copy function's handling of expired keys

When the server is running in replication mode, the deletion of the expired key from the server is controlled by the master server:

• After the master server deletes an expired key, it will explicitly send a DEL command to all slave servers, telling them to delete the expired key.
• When the slave server executes the read command sent by the client, the expired key will not be deleted even if the expired key is encountered, but the expired key will continue to be processed like the unexpired key.
• The slave server will delete the expired key only after receiving the DEL command from the master server.

Reference: redis design and implementation