This paper describes the Redis persistence of two mechanisms: RDB and AOF, as well as key expiration policy: Delete inert and regularly delete, and RDB, AOF and replication handling of expired keys.
RDB
RDB is the first way Redis persistence. There are two commands can be used to generate Redis RDB file, a is the SAVE, the other is BGSAVE.
SAVE blocks the Redis server process, Redis server will block all commands sent by the client execution.
redis> SAVE
OKBGSAVE will derive a child process, the client can continue processing the command execution, but refused clients and BGSAVE SAVE command, delayed BGREWRITEAOF command.
redis> BGSAVE
Background saving startedExecution condition
SAVE command will block the server, it can only be performed manually. BGSAVE can be performed without blocking, so you can save the configuration options for the server at regular intervals automatically performed once.
For example, we can provide the following configuration to the server:
save 900 1
save 300 10
save 60 10000So long as any of the following three conditions is to be executed:
- Server 900 seconds in the database is modified at least once.
- Server within 300 seconds in the database is modified at least ten times.
- Server within 60 seconds of the database is at least 10,000 times changed.
To accomplish this, the server will maintain a record after revision from the last saved number of dirty counter and a lastsave property last saved records.
Cycle serverCron default function will be executed once every 100 milliseconds, it's one job is to check the condition save option settings meets, if so, it will execute BGSAVE command.
document content
RDB file has a plurality of portions, including field handshake 'REDIS' string, the version number, the database, 'EOF' and the check field.
The core part of database fields, database fields, including the field handshake 'SELECTDB', database ID and key-value pair database numbers indicate that this is the first of several databases, and then save the key to the data.
In addition to the types of key data and also may have an expiration time. For a different type of key-value pairs, RDB files to save them in different ways.
RDB file itself is a compressed binary file, each SAVE or BGSAVE will create a new RDB file does not support additional operations.
AOF
AOF is Redis persistence of the second way, when the AOF and RDB open simultaneously, the server will give priority to recover data from AOF, because AOF each time interval shorter records.
RDB and direct recording different key-value pairs, AOF record is the command. Server after a write command finishes, this command will be appended to the end of the server aof_buf buffer and write to the file at an appropriate time. Server creates a pseudo-client reconstruction, in turn execute commands in the file to complete the data load.
Written and synchronizing files
AOF persistence occurs before the end of each event loop, it will block the server. When the persistent calls of the operating system write function, but usually the function will save the data in a memory buffer instead of immediately brush inside the disk. This poses a security problem. To avoid this problem the operating system also provides a synchronization function fsync and fdatasync two mandatory brush plate. We write written called, and the fsync fdatasync called synchronization.
Data server before the end of each event loop according to appendfsync options and synchronization aof_buf written in:
- always: write and synchronization
- everysec: write, if more than one second from the last synchronization, synchronization
- no: just write, when to synchronize determined by the operating system
AOF rewrite
With the passage of time the server is running, the contents of the file AOF will be more and more, the file size will be growing, not only will affect the host computer, but also slow down the time required for data recovery.
AOF rewrite refers to regenerate a AOF AOF file replaces the original file. But not here to rewrite the original file for reading, analysis, or write, but the key-value pairs in the database are translated into commands to re-write the new file.
Rewriting a time-consuming operation, so it Redis to operate in the background, the corresponding instruction is BGREWRITEAOF. The server may also receive new instructions in the rewriting process, so AOF rewrite Redis will maintain a buffer zone, recording a write command during rewrite after rewrite completion of AOF documents appended to the end.
RDB and comparison AOF
RDB advantages :
- RDB is a very compact file, it's smaller, and you can select the persistence of time for doing file backups. For example, daily backups, monthly backups.
- RDB is more friendly to the main process, the parent process only needs to fork out a sub-process, without performing any disk I / O operations.
- RDB speed when recovering large data sets faster than the speed of recovery of AOF.
RDB shortcomings :
- RDB files need to be saved because the state of the entire data set, so it is not an easy operation. So you could save at least 5 minutes before RDB file once, a long time interval.
- Although RDB will persist operation to the child, but to start from scratch every time, when relatively large data sets, fork () can be very time consuming, cause the server to stop processing the client within a certain milliseconds; if the data set is very large, and the CPU time is very tight, then stop this time may even take up to a full second.
AOF advantages :
- AOF using additional embodiment, each write time is very short, and therefore may allow for a shorter interval lasting operation, such as one second.
- AOF document readability is better if you do not accidentally execute a command, as long as the AOF file has not been overwritten, so long as the server stops, remove the piece of AOF file and then restart Redis commands can be.
AOF shortcomings :
- For the same data sets, files generally AOF volume greater than the volume RDB file.
- Redis using fsync degrade performance, resulting in the AOF rate may be slower than RDB.
AOF strengths and RDB, RDB small, fast recovery rate, and can generate a snapshot; AOF higher frequencies, the updated data can be saved. In general, it is recommended to use at the same time.
Redis expires mechanism
Redis is taken regularly delete and delete inert fit for use.
Inert delete refers Redis will check whether the key is expired when accessing a key, if expired, they will be deleted from the database input keys. But can not delete inert cleared up memory, so there is a mechanism for regular Redis deleted.
Deleted regularly is another key to delete outdated way. Redis will expire maintain a dictionary (as shown below), all declared key expiration time will be added to the dictionary. When the periodic operation performed serverCron function, random part of the key check expiration time within a predetermined time, and to delete the expired key.
RDB, AOF and replication process for expired keys
RDB handling of expired keys
RDB file checks the expiration time of each key in the generation, key expiration will not be added to the RDB file.
Loading RDB file, if the server is the primary server, the file is not loaded expired key; if the server from the server, regardless of date or not will be loaded. However, because the master and slave servers when synchronization from the database server will be cleared, so generally speaking, expired key impact on the RDB files are not loaded from the server.
AOF process for expired keys
When AOF file is written, if the database is a key has expired, but it has not been deleted, AOF file will not have any impact on this key. When it is periodically delete inactive or deleted, the program will append a DEL command to the AOF documents show that the key record has been deleted.
When AOF rewrite, and generate RDB files, will filter out the key has expired.
Replication process for expired keys
Master server delete an expired key, will explicitly send a DEL command from the server to all, inform delete the expired key from the server.
Redis 3.2 ago, in order to maintain the master, from the server to delete from the consistency in the implementation of a read command sent by the client, even if the encounter would not expire expired key key, but continue to image processing unexpired key as the disposal of obsolete keys. Only after receiving the master server sent DEL command will delete expired key from the server. After Redis 3.2, when data is read from the node, whether an increase of stale data determination: If the data has expired, is not returned to the client.