We often use caching applications in services, especially Redis. Today, I will talk about Redis data persistence.
There are two main ways of Redis persistence, one is RDB and the other is AOF.
Among them, RDB (Redis DataBase) persistence is performed in a snapshot (snapshotting) mode, in which all data existing at a certain moment is written to the hard disk.
AOF (append-only file) is persisted by appending files. It records each write command in a log file, and re-executes the AOF file command to restore the data after the service restarts.
These two data backup methods can be used at the same time or independently.
RDB snapshot persistence
First, let's look at the RDB snapshot method. This persistence method is mainly divided into manual trigger and passive trigger.
Manual trigger is divided into SAVEand BGSAVE, because SAVE will block the Redis service until the RDB process is completed, so 生产环境我们主要使用BGSAVE操作after the execution of BGSAVE, the Redis process will execute the fork sub-process, and let the sub-process be responsible until it ends automatically.
Passive triggering is due to the save configuration option in our configuration file. If the user sets multiple configuration save configuration options, BGSAVE will be triggered if any one of the conditions is met.
We can look at the following configuration:
1. save 60 10000 # 60s有10000次写入（修改）被动触发BGSAVE
2. save 900 10 # 900s有10次就触发BGSAVE，保存到硬盘（两个满足一个就触发）
3. stop-writes-on-bgsave-error no # 创建快照失败是否继续写命令
4. rdbcompression yes # 是否进行压缩（默认压缩）
5. dbfilename dump.rdb # 生成文件名
Among them, the save configuration requires us to make a reasonable evaluation. Too frequent will waste resources, and too scarce may cause a lot of data loss.
Here we need to know how to save data through snapshot persistence. If the service fails, the user will lose all data changed since the last snapshot.
Of course, if the cached data is less than or only a few GB, there is no problem using snapshots to save the data.
AOF additional persistence
Because RDB snapshot persistence is suitable for small-scale applications or applications where the loss of some data will not cause problems, but in most scenarios, we need to ensure data integrity as much as possible, so here we have AOF.
AOF persistence will write the executed command to the end of the AOF file to record the data changes, so only need to execute the AOF file command in Redis to restore the data of the AOF file.
However, the AOF file will become larger and larger with the continuous increase of commands, so Redis has a rewrite mechanism to compress and control the file size through the rewrite mechanism.
The principle of the rewrite mechanism is to merge multiple commands. For example, if the same inc operation is performed 100 times, then it can be merged into one operation, or a certain data has been added before and then deleted later to delete the previously invalid command, etc.
The AOF rewrite method and RDB trigger are also divided into manual trigger and passive trigger.
Manual trigger mode: bgrewriteaof
passive trigger mode: auto-aof-rewrite-percentage, auto-aof-rewrite-min-sieze
We can look at the following configuration:
appendonly no # 是否进行aof持久化
appendfsync everysec # 每秒同步一次，有always，no选项
no-appendfsync-on-rewrite no # aof重写期间是否同步
auto-aof-rewrite-percentage 100 # 当前文件是上一次大小的两倍的时候触发重写
auto-aof-rewrie-min-size 64mb # 文件最小体积触发重写
The AOF rewrite is the same as the RDB snapshot. It creates a child process, and then the AOF child process rewrites the file. Therefore, the performance and memory problems of the snapshot persistence are the same in AOF because the child process is created.
The AOF method can be relatively more complete to retain data, let us feel more at ease.
Here we need to know the default loading process of Redis after the service restarts, and load the AOF file first. If not, consider loading the RDB file.
At last
Let's briefly summarize the advantages and disadvantages of RDB and AOF:
Advantages of RDB: Compressed binary is suitable for backup, loading and restoring data is faster than AOF.
Disadvantages of RDB: Real-time persistence cannot be achieved, and frequent operations consume resources
Advantages of
AOF: AOF can be persisted in real time through additional commands. Disadvantages: slow to load and restore data
With these two persistence methods, our services can retain data more completely when the service is restarted or down. In addition to the persistence operation, we should also try to back up data files to different servers to avoid serious problems. Time will not lose data.