Redis persistence

Redis offers different persistence options:

RDB persistence at a specified time interval to save that point in time snapshot of the data.
AOF persistence method will write to each server receives record. When the server starts, record of these operations will be executed one by one so as to reconstruct the original data. Consistent write command recording format with Redis agreement to additional ways to save.
Redis can be disabled when persistent, that you can make life-cycle data only exists in the server uptime.
Persistence two ways that can exist at the same time, but when Redis restart, AOF file will be 优先used to reconstruct the data.

The most important thing is to understand the differences between different AOF RDB and persistence.

RDB persistence

Create and load files RDB

There are two commands can be used to generate Redis RDB file, a is the SAVE , the other is bgsave .

SAVE command Redis server process will be blocked until the RDB file is created, the server during the process blocked, the server can not process any command requests:

And SAVE command directly block the server process different approach, BGSAVE command to derive a child process, then the responsibility of the Forbidden City to create RDB file server process (parent process) continues to process commands request:

RDB advantage

RDB file is a compressed binary file, save the Redis data point in time. RDB file is ideal for backup. You can set a time for the RDB file archive, so you can easily when needed to restore the data to a different version.
RDB is ideal for disaster recovery . Single file transfer easily to a remote server.
RDB's performance is very good, when the need for persistence, the main process will fork a child process out, then the persistent work to the child, they will not have the relevant I / O operations, which is said to be above the bgsave .
Compared AOF, in the case of a large amount of data, RDB faster startup.

RDB shortcomings

RDB likely to cause data loss. Assuming that every five minutes to save a snapshot, if Redis for some reason does not work, then produce a snapshot of the data from the last Redis problems during this time will be lost. Above SAVE disadvantages.
RDB use fork () to produce the child for persistence of data, if the data is relatively large might take a point in time, resulting in Redis unavailable for a few milliseconds. If the data is large and the CPU performance is not very good when out of service time even more. Above BGSAVE disadvantages.

File path and name

The default is a directory named Redis will the current snapshot files are stored dump.rdbfiles. To modify the storage path and name of the file, you can modify the configuration file redis.confto achieve:

# RDB文件名，默认为dump.rdb。
dbfilename dump.rdb

# 文件存放的目录，AOF文件同样存放在此目录下。默认为当前工作目录。
dir ./

Save points (RDB enabled and disabled)

You can configure the save point, so if the Redis M times changed in every N seconds after the data is saved snapshot file. For example, the following configuration represents a saving point every 60 seconds, if the data has more than 1000 times of change, Redis will automatically save a snapshot of the file:

save 60 1000

Can set up multiple save points, Redis configuration file to the default settings saved three points:

# 格式为：save <seconds> <changes>
# 可以设置多个。
save 900 1 #900秒后至少1个key有变动
save 300 10 #300秒后至少10个key有变动
save 60 10000 #60秒后至少10000个key有变动

If you want to disable saved snapshot feature can be configured to achieve by commenting out all "save", or add the following configuration after the last "save" configuration:

save ""

Error Handling

By default, if Redis fail when a snapshot of the background, it will stop receiving the data, the purpose is to let users know that the data is not persistent success. But if there are other ways you can monitor the status of Redis and persistence, you can prohibit this feature off.

stop-writes-on-bgsave-error yes

data compression

The default Redis will be used LZFto compress the data. If you want to save CPU performance points, you can disable the compression feature off, but the data set will not play than when compressed.

rdbcompression yes

Data validation

From the beginning of RDB version 5, and a CRC64validation code will be placed at the end of the file. So we can guarantee the integrity of the document, but when you save or load the file will lose some performance (about 10%). If you want to pursue higher performance, it can disable the swap, so when writing the file checksum will use 0an alternative, load time to see 0it will skip the check.

rdbchecksum yes

AOF persistence

Snapshot is not very reliable. If your computer suddenly goes down, or power off, or accidentally kill the process, the latest data will be lost. The AOF document provides a more reliable and persistent manner. Whenever Redis commands received will modify the data set, the command will be appended to the AOF file when you restart Redis, AOF in the command will be executed once again, to rebuild the data.

Use AOF persistent need to set the synchronization options to synchronize the timing of command to ensure that files on the disk. This is because the file for writing b does not immediately synchronized to the content on the disk, but the first store into the buffer, and then determined by the operating system when synchronized to disk. The following synchronization options:

Options	Synchronization frequency
always	Each write command are synchronized
everysec	Sync once per second
no	Let the operating system to decide when to synchronize

advantage

Reliable than RDB. You can make different fsync strategy: without fsync, fsync every second and every query fsync. Fsync default is once per second. This means that you lose up to one second of data.
AOF additional log file is a plain file. Even encountered a sudden power failure, the positioning of the logs will not appear or corruption issues. Even if for some reason (such as a disk full) command only half-written to a log file, we can also use redis-check-aofthis tool very simple to repair.
When AOF file is too large, Redis will automatically be rewritten in the background. Rewrite very safe, because rewrite is carried out on a new file, while Redis will continue to append the old file data. The minimum set of operational command will write a new file to rebuild the current data set. When a new file finished weight, the Redis old and new file will be switched on and starts writing data to the new file.
AOF easy to understand the operation command to format one after another in the file is saved, it is easy to lead out for restoring data. For example, we are not careful with the FLUSHALLcommand to brush off all the data, as long as the file is not overridden, we can service stopped, the last piece of the command to delete, and then restart the service, so you can put the brush back data recovery .

Shortcoming

Under the same set of data, the size of the file AOF generally larger than the RDB file.
In some fsync strategy, AOF speed will be slower than RDB. Fsync usually set once per second will be able to get higher performance, but in the case of prohibited fsync speed can reach the level of RDB.
In the past we have found some very rare BUG result in data usage AOF reconstructed with the original data inconsistencies.

Enable AOF

The configuration item appendonlyto yes:

appendonly yes

File path and name

# 文件存放目录，与RDB共用。默认为当前工作目录。
dir ./

# 默认文件名为appendonly.aof
appendfilename "appendonly.aof"

reliability

You can configure Redis call frequency fsync, there are three options:

Whenever a new command added to AOF when calling fsync. The slowest, but the most secure.
fsync once per second. Speed (2.4 version almost like snapshots speed), security is good (up to 1 second loss of data).
Never fsync, handed over the system to deal with. The fastest way, but security in general.

Fsync recommended once per second way (the default mode), because it was fast, security is also good. Configuration is as follows:

# appendfsync always
appendfsync everysec
# appendfsync no

Log rewrite

With the increasing write operation, AOF file will grow. For example, you increment a counter 100, then the end result is the final result of incremental data set counter, but AOF file will actually complete record 100 times this operation. To recover this record and in fact, only one command on the line, that is to say that 100 AOF file command can actually be streamlined into one. So Redis support such a feature: without service interruption rebuild AOF files in the background.

It works as follows:

Redis calls fork (), generates a child process.
The new child process AOF written to a temporary file.
Main ongoing process to write the new changes in the memory buffer, but also to these new changes written in old AOF, so even if the failure can be rewritten to ensure data security.
After the completion of the process of rewriting the file, the main process will get a signal, and then the memory of buffer is added to the new child process resulting in AOF.
Redis

We can set the conditions by logging rewrite configuration:

# Redis会记住自从上一次重写后AOF文件的大小（如果自Redis启动后还没重写过，则记住启动时使用的AOF文件的大小）。
# 如果当前的文件大小比起记住的那个大小超过指定的百分比，则会触发重写。
# 同时需要设置一个文件大小最小值，只有大于这个值文件才会重写，以防文件很小，但是已经达到百分比的情况。

auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb

To disable the automatic log function of rewrite, we can put the percentage is set to 0:

auto-aof-rewrite-percentage 0

Redis can automatically log rewritten more than 2.4, the previous version will need to manually run BGREWRITEAOF this command.

Data corruption repair

If for some reason (such as a server crash) AOF file is damaged, resulting in Redis can not be loaded, it can be repaired in the following ways:

AOF backup file.
Use redis-check-aofcommand to repair the original AOF file:
```
$ redis-check-aof --fix
```
You can use diff -uthe command look differences between the two documents.
Using the restored file restart Redis service.

Switching from AOF to RDB

Just say here that Redis> 2.2 version of the Method =:

A new backup dump.rdbfile, and put the backup file in a safe place.

Run the following two commands:

$ redis-cli config set appendonly yes
$ redis-cli config set save ""

Ensure that the data are consistent with the pre-handover.
To ensure that data is properly written AOF file.

The second command is used to disable the persistent mode RDB, but this is not necessary, because you can enable two persistent simultaneously.

I remember the configuration file redis.conffor editing enabled AOF, because the command line to modify the configuration will be invalid after the restart Redis.

Backup

Recommended backup methods:

Create a scheduled task, create a snapshot every hour and every day, stored in different folders.
Timing task runs, too old to delete files. For example, to retain only 48-hour snapshot created by the hour and one to two months by the day of the snapshot was created.
Once a day to ensure that the transfer snapshot files to a place outside the data center to save, save at least not in Redis server service is located.

reference

Huang Jianhong. Redis design and implementation of [M]. Machinery Industry Press, 2014.
https://redis.io/topics/persistence