Redis persistence Summary

RDB

RDB persistence is through snapshots (snapshotting) completed, when certain conditions are met, Redis memory of all the data in binary form to generate a copy and store on your hard drive.

Trigger mechanism

• save command: Redis server is currently blocked until the process is complete RDB, for example relatively large memory for a long time can cause obstruction, online environment is not recommended
• bgsave command: Redis process of implementation fork operation to create a child process, RDB lasting process responsible for the child process ends automatically after completion. Blocking occurs only in the fork phase, generally a very short time
• Use the save configuration, such as "save mn". When the modification indicates the presence of n times m second data set, automatically trigger bgsave
• If the total amount of the copy operation performed from the node, the master node is performed automatically bgsave RDB generated from the node sends the file
• Executing debug reload command to reload Redis, it will automatically trigger the save operation.
• When you execute the shutdown command and flushall, if not open AOF persistence function is performed automatically by default bgsave

bgsave operational processes

1. Bgsave execute commands, Redis parent process child process to determine whether there is currently being implemented, such as RDB / AOF child process, if there is a direct return bgsave command
2. Parent process fork operation to create a child process, the parent process fork operation will block view latest_fork_usec options via info stats command, you can get a fork recent time-consuming operation, the unit is the role of microseconds .Fork is a copy of the current process the same process. All data (variables, environment variables, program counter, etc.) and the value of the new process are consistent with the original process, but is a new process, and as a child process of the original process
3. After the completion of the parent process fork, bgsave command returns "Background saving started" information and will not clog the parent process, you can continue to respond to other commands
4. RDB child process creates a file (Rdb is dump.rdb saved file), generates a temporary memory snapshot file according to the parent process, the original file is replaced after the completion of atoms. Lastsave command execution time of the last generation can get RDB, the corresponding info statistics rdb_last_save_time options
5. Process sends a signal to indicate the completion of the parent process, the parent process to update statistics

Data recovery

Rdb file will be moved to redis data directory and start the service, redis file will automatically load the data into memory.

After the RDB Redis starts, it reads the snapshot file, the data is loaded from the hard disk into memory. The amount of data with different structures and server performance, this time is different. Typically ten million records a character string type of the keys, the size of 1GB snapshot file is loaded into memory takes 20 to 30 seconds. By RDB way for persistence, once

Redis abnormal exit, it will lose the last snapshot of all data changes in the future.

Advantage

RDB is a compact compressed binary file, on behalf of Redis data snapshot at a certain point in time. Ideal for backup, the whole amount of replication scenarios. For example bgsave backup performed every 6 hours, and copy the file to the remote machine or RDB file system (e.g. HDFS), for disaster recovery (less demanding on data integrity and consistency)

Redis load RDB to recover data much faster than AOF way (for large-scale data recovery)

Disadvantaged

RDB way no way to do real-time data persistence / second-level persistence. Bgsave because each run should be executed fork operation to create a child process belongs to the heavyweights operation (data in memory is a clone, approximately 2-fold expansion need to be considered), frequently perform high cost (affects performance)

RDB file saved using a specific binary format, Redis version of evolution have multiple versions of RDB format, there is the old version of Redis service can not issue a new version of RDB-compatible format (incompatible version)

A certain time interval to do a backup, so if redis accidentally fall down, they would lose all changes after the last snapshot (data loss there)

Stop RDB save

redis-cli config set save ""

AOF

AOF (append only file) recorded in a separate log of each write command, and then re-execute the purpose of AOF file command to achieve data recovery restart. AOF main role is to solve the real-time data persistence, persistence Redis is now the mainstream way. Popular thing to understand is in the form of a log to record each write operation, Redis executed all write commands recorded only append files to form, beginning redis starts, it reads the file to reconstruct the data.

How to open

Set Configuration: appendonly yes, the default is not open.

AOF name of the configuration file: appendfilename, the default file name is appendonly.aof

3 kinds of flashing mode

• appendfsync always # each time it receives a write command to force an immediate write to the disk, is the best guarantee of complete persistence, but also the speed of the slowest, is generally not recommended.
• appendfsync everysec # once per second compulsory written to disk, performance and persistence has done a good compromise, it is subject to the recommended way.
• appendfsync no # OS written entirely dependent, usually about 30 seconds, the best performance but does not guarantee the most persistent, is not recommended.

AOF workflow

1. All the write command will be appended to the aof_buf buffer;
2. aof_buf synchronization operation to the hard disk buffer made according to the corresponding policy;
3. With the growing need for regular file AOF rewrite of the AOF files, to achieve the purpose of compression;
4. When the Redis server restarts, you can load AOF file for data recovery.

Rewriting mechanism

With the command continues to write AOF, the file will be larger in order to solve this problem, Redis AOF rewrite mechanism introduced compressed file size. AOF file is to rewrite the data in Redis process into a write command to the new AOF file synchronization process. Popular understanding AOF file using append mode, the file will be much more to avoid such a situation occur, the new rewrite mechanism, when the size of the AOF file exceeds the threshold set, Redis will start AOF documents content compression, leaving only the minimum set of instructions may recover the data can use the command bgrewriteaof.

Principle: AOF file continues to grow and is too large, it will fork out a new process to rewrite the file (also the first to write temporary files and finally rename), in-memory data traversing the new process, each record has one of the Set statement. Rewrite operation aof files and does not read the old aof file, but the contents of the entire database in memory with the command to rewrite a new way of aof file, which is somewhat similar point and snapshots

AOF rewrite process can be triggered manually and automatically triggers:

• Manual trigger: a direct call bgrewriteaof command.
• Auto: The timing is determined automatically trigger auto-aof-rewrite-min-size and auto-aof-rewrite-percentage parameters. When Redis will record the last time when the size of the AOF rewrite the default configuration when the AOF file size is doubled and the file size is greater than the last rewrite trigger 64M

Data recovery

When the data file corruption How to fix

For AOF file format error, first backup, then using redis-check-aof - fix command to fix, using comparative data differences diff-u after the repair, find lost data, you can manually modify some completions. AOF files may exist at the end it is incomplete, such as a sudden power failure cause the machine AOF end of the file write command incomplete. Redis provides aof-load-truncated configured to be compatible with our case, enabled by default. When loading AOF, when faced with this problem is ignored and continue to boot.

Performance Summary

1. Because RDB file only as a back-up purposes, it is recommended only in Slave persistent RDB file, and as long as 15 minutes backup once is enough, leaving only save 900 1 this rule.
2. If Enalbe AOF, benefit is in the worst case it will only lose no more than two seconds of data, startup scripts simpler just load your own AOF files on it. First, the cost of bringing sustained IO, and second, AOF rewrite the last rewrite new data generated in the process of blocking the new file is written to cause almost inevitable. As long as the hard disk license, you should try to reduce the frequency of AOF rewrite. AOF override the default value on the basis 64M size is too small, it can be set to more than 5G. The default size is more than 100% of the original size can be changed to an appropriate override value.
3. If you do not Enable AOF, alone Master-Slave Replication can achieve high availability. Can save a lot of IO system also reduces the volatility of the rewrite time. If the price is Master / Slave simultaneously drained, I lost ten minutes of data, but also the startup script to compare two Master / RDB file Slave in loading the newer one. Sina microblogging on the selection of this architecture

Reference material

https://blog.csdn.net/weixin_39040059/article/details/79120416
https://blog.csdn.net/weixin_39040059/article/details/79120444