Redis persistence
Redis is an in-memory database. When the redis server restarts and the computer restarts, the data will be lost. We can persist the data in the redis memory to a file on the hard disk. Currently, the storage mechanisms supported by Redis are RDB and AOF.
RDB mechanism
RDB persistence is the process of generating a snapshot of the current process data and saving it to the hard disk. The process of triggering RDB is divided into manual triggering and automatic triggering.
Trigger mechanism:
1. Manual triggering through save and bgsave commands
- save command: Block the current Redis server until the RDB process is completed. For instances with large memory, it will cause long-term blocking. It is not recommended to use it in an online environment.
- bgsave command: The Redis process executes the fork operation to create a child process. The RDB persistence process is the responsibility of the child process and ends automatically after completion. Blocking only occurs in the fork phase, and the time is generally very short (microsecond level).
2. The configuration file redis.windwos.conf that automatically triggers redis through the configuration file has the following default contents:
#In 900 seconds, if at least one key is modified, perform the persistence operation
save 900 1
#In 300 seconds, if at least 10 keys are modified, perform the persistent operation
save 300 10
#In 60 seconds, at least there are After 1000 keys have been modified, the persistence operation is performed
save 60 10000
Process description:
1. The parent process performs a fork operation to create a child process
2. The child process creates an RDB file, generates a temporary snapshot file according to the memory of the parent process, and atomically replaces the original file after completion.
3. The process sends a signal to the parent process to indicate completion, and the parent process updates the statistics.
The main process does not perform any io operations in the whole process, which ensures performance. If large-scale data recovery is performed, both RDB and AOP can perform data recovery. RDB data recovery integrity is not sensitive, and RDB is more efficient. Data may be lost. RDB is used by default. Generally, this configuration does not need to be modified.
RDB file processing:
The RDB file is saved in the directory specified by the dir configuration, and the file name is specified by the dbfilename configuration. The default is a dump.rdb file (it is best to back up the dump.rdb file in the production environment). It can be executed dynamically through execution config set dir {newDir}and config set dbfilename {newFileName}runtime, and will be saved to a new directory when it is next run.
If the RDB file loaded by redis refuses to start when it is damaged, you can use the redis-check-dumptools provided by Redis to detect the RDB file and get the corresponding error report.
RDB advantages:
- RDB is a compact compressed binary file that represents a snapshot of Redis data at a point in time. Great for backups: the parent process handles user requests normally, and fork branches a child process for backup.
- It is suitable for large-scale data recovery. If the server is down, do not delete the rdb file. The restart is naturally in the directory, and it will be automatically read, and the data recovery speed is much faster than the AOF method.
Disadvantages of RDB:
- RDB cannot achieve real-time persistence, storage requires a certain time interval, and you can modify the settings yourself. If redis crashes unexpectedly, the last modified data will be lost.
- When forking a process, it will occupy a certain amount of memory space.
- RDB is saved in a specific binary format, and there may be incompatibility between old and new versions.
AOF mechanism
AOF (append only file) persistence: Record each executed command (except read operations) in an independent log, and re-execute the commands in the AOF file to restore the file when restarting. (The default of aof is that the file is infinitely appended, and the size will continue to expand. If the file is too large, it will take a long time to write)
Turning on AOF requires us to set the configuration: appendonly yes , which is not turned on by default. The file name is set by appendfilename configuration, the default file name is appendonly.aof .
Implementation process:
- Command write (append): All commands will be appended to aof_buf (buffer).
- File synchronization (sync): The AOF buffer synchronizes with the hard disk according to the corresponding policy.
- File rewrite (rewrite): As the file becomes larger and larger, the AOF file needs to be rewritten regularly to achieve the purpose of compression.
- Restart loading (load): When restarting, the AOF file can be loaded for data recovery
AOF file corruption:
When loading a corrupt AOF file it refuses to start and prints a log. We can back up the files first, and then use the redis-check-aof --fix command to repair them. After repairing, use diff -u to compare the differences in the data to find out the lost data. Some of them can be repaired manually.
AOF may have incomplete ending preservation. For example, when the machine is powered off, the ending command is not fully written. Redis provides aof-load-truncated configuration to be compatible with this situation, which is enabled by default. This will continue to start if there is a problem with loading, and print a warning log.