Redis persistence knowledge -RDB + AOF

EDITORIAL

Redis is a key-value high-performance distributed database. It supports a variety of data structures can be applied to the cache, and the like under a variety of scenarios queue

As already written a cluster approach and penetrate the avalanche on the Redis:

Redis + three clusters way to penetrate and avalanches prevention and resolution

In this paper, in-depth understanding of RDB and AOF both logic and persistence of principle, like a friend little attention and forward, not written your big brother is also a lot of criticism and correction, in order to make progress together, thank you.

RDB principle

RDB persistent storage that is the presence of data in memory redis saved as a snapshot of the local disk

RDB persistent strengths and weaknesses

Advantage

• Data snapshot file implementation, full backup, to facilitate the transmission of data. For example, we need to transfer the backup file on the server A to server B above, can be copied directly to the file rdb.
• File is a compressed binary file, load a data file when restarting services, faster than aof way.

Disadvantaged

• rbd encrypted binary file format to store, due to compatibility issues between the various versions of Redis also led rdb by the version compatibility issues unusable yet other Redis versions.
• Timeliness is poor, easily lead to incomplete data, because rdb is not real-time backup when a certain period of time Redis service is abnormal, loss of memory data, data from this period can not be restored, so easily lead to loss of data.

Manual Backup

Manual backup by the save command and bgsave command. save a synchronous blocking, and bgsave is non-blocking (blocking fork actually occurs in the child's). Therefore, in the actual process we use is mostly bgsave backup command

Automatic Backup

1. Save mn modify configuration item means that the command is executed n times within m seconds to backup.
2. When a duplication request transmitted from the server Redis item master server, the master file is generated using the bgsave rbd command, and then transmitted to the server.
3. When executing debug reload command will use the save command rdb file.
4. When using the shutdown command to turn off the service, if not enabled aof way to achieve persistence bgsave way of doing persistence will be adopted at the same time can be added later shutdown backup parameters [nosave | save]

bgsave persistent storage implementation principle

1. Bgsave execute commands, Redis parent process child process to determine whether there is currently being executed, if there is a direct return.
2. Father process fork a child process (process fork in the cause of obstruction case), this process can use the info stats command to view latest_fork_usec option to view the last trumpet fork operation time, the unit is subtle.
3. After the completion of the parent process fork, it returns information prompt Background saving started, then fork to unblock.
4. fork a child process begin running based on the parent's temporary memory data snapshot file, and then replace the original file. Use lastsave command to view the last generation time rdb, the corresponding info in rdb_last_save_time options.
5. When the backup is complete to send information to the parent process is completed

RDB file saving process

1. redis calls fork, now with the child and the parent.
2. The parent process to continue processing client requests, the child process is responsible for the contents of the memory is written to a temporary file. Since the replication mechanism (copy on write) write os of the parent and child share the same physical page, when the parent process to handle write requests os parent process will create a copy of the page you want to modify, rather than writing pages that are shared. Therefore, data in the address space of the child process is a snapshot of the entire database fork moment. ,
3. When the child process to write temporary files snapshot is completed, replace the original file with the temporary snapshot file, and then the child process exits

fork operation will be blocked, resulting in Redis read and write performance. We can control the individual maximum memory Redis instance, to reduce as much as possible the time consumption Redis fork; or triggered automatically controlling the frequency of the fork reduces the number of

AOF principle

AOF view the entire process generally can be divided into two steps, step is to write real-time command (if it is appendfsync everysec configuration, there 1s loss), the second step is a rewrite of the aof file

1. Write command
2. Appended to aof_buf
3. Aof synced to disk

AOF advantages and disadvantages

advantage

• Multiple file write (fsync) strategy.
• Real-time data storage, data integrity and strong. Even if some data loss, but also to develop a good strategy up to within one second of data loss.
• Readable, since the data is used to store text protocol format, may have a direct view of the operation command, but can also rewrite command manually.

Shortcoming

File size is too large, loading speed slower than rbd due to aof the log record is redis operation, some invalid, simplify operations will be recorded, resulting in aof file is too large. But the way the policy can be overridden by file optimize.

AOF persistent storage implementation principle

1. redis write command, the write command at this time will redis aof_buf change from region.
2. The data buffer written to the log file based backup strategy implementation.
3. When aof increasingly large files, will be implemented according to rewrite aof our configuration strategies to achieve compressed files, reduce the volume.
4. When redis restarts, to rewrite load aof files, to achieve the purpose of data recovery.

Select AOF or RDB data persistence

1. For different situations to choose, we recommend using a combination of two ways.
2. For data security, high integrity requires aof way.
3. For less critical data can be used rdb way.
4. For the full backup data, to facilitate data backup rdb manner may be employed.

This article ends, like a friend little bit of attention and praise, thanks! ! !