1. What persistence mechanisms does Redis provide
1). RDB persistence:
This mechanism refers to writing a snapshot of the data set in memory to disk within a specified time interval.
2). AOF persistence:
This mechanism will record each write operation processed by the server in the form of a log. At the beginning of the Redis server startup, the file will be read to rebuild the database to ensure that the data in the database is complete after startup. of.
3). No persistence:
We can disable the persistence function of the Redis server by configuration, so that we can treat Redis as an enhanced version of memcached.
4). Apply AOF and RDB at the same time.
5). If you want to enable AOF mode, modify the Redis configuration file redis.conf.
# 相关配置
appendonly yes #开启AOF模式 原文1
appendfilename "appendonly.aof" #保存数据的AOF文件名称 原文1
# appendfsync always
appendfsync everysec #fsync模式 原文2
# appendfsync no
no-appendfsync-on-rewrite no #原文3
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb #原文4
aof-load-truncated yes #原文5The official documentation explains the following:
原文1：
By default Redis asynchronously dumps the dataset on disk. This mode is good enough in many applications, but an issue with the Redis process or a power outage may result into a few minutes of writes lost (depending on the configured save points).
The Append Only File is an alternative persistence mode that provides much better durability. For instance using the default data fsync policy (see later in the config file) Redis can lose just one second of writes in a dramatic event like a server power outage, or a single write if something wrong with the Redis process itself happens, but the operating system is still running correctly.
AOF and RDB persistence can be enabled at the same time without problems. If the AOF is enabled on startup Redis will load the AOF, that is the file with the better durability guarantees.
译文1：
Redis默认采用异步的方式将数据存放到磁盘上，这个模式对大部份应用来说是足够好的，但是在Redis进程或电源发生故障的情况下，可能会造成小部份的数据丢失，这取决于配置的保存时间点。
Appendonly是一种能够提供非常好的持久化的模式，例如使用默认的Fsync方案，Redis能在发生服务器电源故障或操作系统仍然正常运行但Redis进程莫名挂掉的情况下，只丢失1秒的数据。
AOF与RDB模式可以同时启用，这并不冲突。如果AOF是可用的，那Redis启动时将自动加载AOF，这个文件能够提供更好的持久性保障。
原文2：
The fsync() call tells the Operating System to actually write data on disk instead of waiting for more data in the output buffer. Some OS will really flush data on disk, some other OS will just try to do it ASAP.
Redis supports three different modes:
no: don’t fsync, just let the OS flush the data when it wants. Faster.
always: fsync after every write to the append only log. Slow, Safest.
everysec: fsync only one time every second. Compromise.
The default is “everysec”, as that’s usually the right compromise between speed and data safety. It’s up to you to understand if you can relax this to “no” that will let the operating system flush the output buffer when it wants, for better performances (but if you can live with the idea of some data loss consider the default persistence mode that’s snapshotting), or on the contrary, use “always” that’s very slow but a bit safer than everysec.
If unsure, use “everysec”.
译文2：
fsync()调用告诉操作系统将数据真实的写入磁盘而不是放到缓冲区中，一些操作系统会真实的执行请求，还有一些操作系统只会尽力的尝试。
Redis支持3种不同的模式：
no：不即时同步，由操作系统控制何时刷写到磁盘上，这种模式速度最快；
always：每次只写日志，速度较慢，但最安全；
everysec：每秒钟同步一次，折中的方案。
默认的模式是“everysec”，它通常是在速度和数据安全之间折中的方法。如果你可以控制操作系统在Redis需要的时候去刷写缓冲区那可以使用“no”模式，能够提供更好的性能（但如果你能接受一些数据丢失，可以考虑默认的持久化模式–快照方式），相反，使用“always”模式很慢，但是它比“everysec”模式要安全一点。
如果不确定，就使用“everysec”。
原文3：
When the AOF fsync policy is set to always or everysec, and a background saving process (a background save or AOF log background rewriting) is performing a lot of I/O against the disk, in some Linux configurations Redis may block too long on the fsync() call. Note that there is no fix for this currently, as even performing fsync in a different thread will block our synchronous write(2) call.
In order to mitigate this problem it’s possible to use the following option that will prevent fsync() from being called in the main process while a BGSAVE or BGREWRITEAOF is in progress.
This means that while another child is saving, the durability of Redis is the same as “appendfsync none”. In practical terms, this means that it is possible to lose up to 30 seconds of log in the worst scenario (with the default Linux settings).
If you have latency problems turn this to “yes”. Otherwise leave it as “no” that is the safest pick from the point of view of durability.
译文3：
当使用AOF的fsync方案设置为“always”或“everysec”时，后台的存储进程会执行大量的磁盘I/O操作，在一些Linux架构中，Redis在fsync()调用时可能会阻塞很久。这个问题当前并没有修复，即使是在一个不同的线程执行fsync也将会阻塞我们的同步写调用。
为了缓解这个问题，可以使用以下选项，它将会在有一个BGSAVE或BGREWRITEAOF正在运行时，阻止主进程调用fsync()。
这意味着有另一个子进程在存储时，Redis的持久性等同于“appendfsync none”。在实践中，意味着在最坏的情况下它可能丢失多达30秒的日志（默认的Linux设置）。
如果你有潜在的问题需要更改它为“yes”。否则从持久性的观点来看“no”是最安全的选择。
原文4：
Automatic rewrite of the append only file.
Redis is able to automatically rewrite the log file implicitly calling BGREWRITEAOF when the AOF log size grows by the specified percentage.
This is how it works: Redis remembers the size of the AOF file after the latest rewrite (if no rewrite has happened since the restart, the size of the AOF at startup is used).
This base size is compared to the current size. If the current size is bigger than the specified percentage, the rewrite is triggered. Also you need to specify a minimal size for the AOF file to be rewritten, this is useful to avoid rewriting the AOF file even if the percentage increase is reached but it is still pretty small.
Specify a percentage of zero in order to disable the automatic AOF rewrite feature.
译文4：
自动重写append only文件。
当AOF日志的大小根据指定的百分比增长时，Redis会暗中调用BGREWRITEAOF去自动重写日志文件。
工作原理：Redis记忆AOF文件最后一次重写的大小（如果重启后没有重写发生，AOF的大小在启动时会被使用）。
基本大小对比当前大小。如果当前大小比指定的百分比大，触发重写。并且你要为AOF文件指定一个最小的尺寸去重写，这对于避免重写AOF文件是有用的，即使达到了百分比增长率但它仍然是非常小的。
指定百分比为0以便禁用自动AOF重写。
原文5：
An AOF file may be found to be truncated at the end during the Redis startup process, when the AOF data gets loaded back into memory.
This may happen when the system where Redis is running crashes, especially when an ext4 filesystem is mounted without the data=ordered option (however this can’t happen when Redis itself crashes or aborts but the operating system still works correctly).
Redis can either exit with an error when this happens, or load as much data as possible (the default now) and start if the AOF file is found to be truncated at the end. The following option controls this behavior.
If aof-load-truncated is set to yes, a truncated AOF file is loaded and the Redis server starts emitting a log to inform the user of the event. Otherwise if the option is set to no, the server aborts with an error and refuses to start. When the option is set to no, the user requires to fix the AOF file using the “redis-check-aof” utility before to restart the server.
Note that if the AOF file will be found to be corrupted in the middle the server will still exit with an error. This option only applies when Redis will try to read more data from the AOF file but not enough bytes will be found.
译文5：
可能发现一个AOF文件在Redis启动过程中被截断，当AOF数据被加载回内存时。
这可能发生在系统中Redis运行崩溃，尤其是挂载一个EXT4文件系统而没有使用“data=ordered”选项时（但是这不会发生在Redis自身崩溃或中断而操作系统仍然正常运行的时候）。
当有一个错误发生时Redis要么退出，要么加载尽可能多的数据（当前默认）启动，如果AOF文件最后发现被截断。以下选项控制这个行为。
如果aof-load-truncated被设置为“yes”，一个被截断的AOF文件将会被加载，Redis服务启动发出一个日志告知用户这个事件。如果这个选项设置为“no”，服务器中止一个错误并拒绝启动。当选项被设置为“no”，用户需要在重启服务之前使用“redis-check-aof”功能确定AOF文件。
需要注意的是如果AOF文件被发现是损坏的，那服务器仍然会以一个错误退出。这个选项仅适用于Redis希望读取更多的数据但是没有发现足够的字节时。
Second, the advantages and disadvantages of the RDB mechanism
What are the advantages of RDB?
1). Once this method is adopted, then your entire Redis database will contain only one file, which is perfect for file backups. For example, you might plan to archive the last 24 hours of data every hour, and also archive the last 30 days of data every day. Through such a backup strategy, once the system fails catastrophically, we can restore it very easily.
2). For disaster recovery, RDB is a very good choice. Because we can very easily compress a single file and then transfer it to other storage media.
3). Maximize performance. For the Redis service process, when starting persistence, the only thing it needs to do is to fork the child process, and then the child process will complete the persistence work, which can greatly avoid the service process from performing IO operations.
4). Compared with the AOF mechanism, if the data set is large, the startup efficiency of RDB will be higher.
What are the disadvantages of RDB?
1). If you want to ensure high availability of data, that is, to avoid data loss to the greatest extent, then RDB will not be a good choice. Because once the system goes down before the scheduled persistence, the data that has not been written to the disk before will be lost.
2). Since RDB assists in completing data persistence through fork child processes, if the data set is large, it may cause the entire server to stop serving for hundreds of milliseconds, or even 1 second.
3. Advantages and disadvantages of AOF mechanism
What are the advantages of AOF?
1). This mechanism can bring higher data security, that is, data persistence. Three synchronization strategies are provided in Redis, namely synchronization per second, synchronization per modification, and non-synchronization. In fact, the synchronization is done asynchronously every second, and its efficiency is also very high. The difference is that once the system goes down, the data modified within this second will be lost. And every modification synchronization, we can think of it as synchronization persistence, that is, every data change that occurs will be recorded to disk immediately. Predictably, this approach is the least efficient. As for no synchronization, no need to say much, I think everyone can understand it correctly.
2). Since the mechanism uses the append mode for the log file writing operation, even if there is a downtime during the writing process, the existing content in the log file will not be destroyed. However, if we only write half of the data in this operation and the system crashes, don't worry, we can use the redis-check-aof tool to help us solve the problem of data consistency before Redis starts next time.
3). If the log is too large, Redis can automatically enable the rewrite mechanism. That is, Redis continuously writes modified data to the old disk file in append mode, and Redis also creates a new file to record which modification commands are executed during this period. Therefore, data security can be better ensured when rewrite switching is performed.
4). AOF includes a well-formed, easy-to-understand log file for recording all modification operations. In fact, we can also complete the reconstruction of the data through this file.
What are the disadvantages of AOF?
1). For the same number of datasets, AOF files are usually larger than RDB files.
2). Depending on the synchronization strategy, AOF tends to be slower than RDB in operating efficiency. In short, the efficiency of the synchronization strategy per second is relatively high, and the efficiency of the synchronization disable strategy is as efficient as that of RDB.
Fourth, the specific configuration implementation method introduction
1. Snapshotting:
By default, Redis dumps a snapshot of the dataset into the dump.rdb file. In addition, we can also modify the frequency of Redis server dump snapshots through the configuration file. After opening the 6379.conf file, we search for save and see the following configuration information:
save 900 1 #After 900 seconds (15 minutes), If at least 1 key changes, dump the memory snapshot.
save 300 10 #After 300 seconds (5 minutes), if at least 10 keys have changed, dump the memory snapshot.
save 60 10000 #After 60 seconds (1 minute), if at least 10000 keys have changed, dump the memory snapshot.
2. The mechanism of Dump snapshot:
1). Redis first fork the child process.
2). The child process writes the snapshot data to the temporary RDB file.
3). When the child process completes the data writing operation, replace the old file with the temporary file.
3. AOF file:
As mentioned above, the snapshot timing dump mechanism of RDB cannot guarantee good data durability. If our application is really concerned about this, we can consider using the AOF mechanism in Redis. For the Redis server, the default mechanism is RDB. If you need to use AOF, you need to modify the following entries in the configuration file:
change appendonly no to appendonly yes
From now on, Redis will receive a data modification command every time After that, it will be appended to the AOF file. At the next restart of Redis, the information in the AOF file needs to be loaded to build the latest data into memory.
4. AOF configuration:
There are three synchronization methods in the Redis configuration file, they are:
appendfsync always #AOF file will be written every time a data modification occurs.
appendfsync everysec #Sync every second, this policy is the default policy of AOF.
appendfsync no #Never sync. Efficient but data will not be persisted.
5. How to fix corrupt AOF files:
1). Make an extra copy of the existing damaged AOF file.
2). Execute the "redis-check-aof --fix <filename>" command to fix the corrupt AOF file.
3). Restart the Redis server with the repaired AOF file.
6. Redis data backup:
In Redis, we can back up running Redis data files online by copying. This is because the RDB file cannot be modified once it is generated. Redis dumps the latest data into a temporary file every time, and then uses the rename function to atomically rename the temporary file to the original data file name. So we can say that copying data files at any time is safe and consistent. In view of this, we can regularly backup Redis data files by creating a cron job, and copy the backup files to a safe disk medium.
Five, in-memory database Redis persistence summary
Redis can obtain a copy of the data in memory at a certain point in time by creating a snapshot. After a snapshot is created, users can back up the snapshot, copy the snapshot to another server to create a copy of the server with the same data, and leave the snapshot in place for use when the server is restarted.
There are two commands that can be used to generate RDB files, one is SAVE and the other is BGSAVE.
When only using snapshot persistence to save data, if the system does crash, users will lose all data changed since the last snapshot was taken. Therefore, snapshot persistence is only suitable for applications where losing some data is not a problem.
SAVE
Features: The SAVE command will block the Redis server process until the RDB file is created. During the blocking period of the server process, the server cannot process any command requests.
Disadvantage: The server cannot accept other requests during persistence.
BGSAVE
Features: The BGSAVE command will spawn a child process, and then the child process is responsible for creating the RDB file, and the server process continues to process the command request.
Disadvantage: The time it takes to create a child process will increase with the memory occupied by Redis.
AOF persistence
AOF persistence will write the executed write commands to the end of the AOF file to record the changes in the data. Therefore, Redis can restore AOF as long as all the write commands contained in the AOF file are re-executed from the beginning to the end. The dataset recorded by the file.
When setting the sync frequency, there are three options:
| Options | Sync frequency |
|---|---|
| always | Each Redis write command must be written to the hard disk synchronously, but doing so will occupy the memory owned by Redis and seriously reduce the speed of Redis |
| everysec | Perform a sync every second to explicitly sync multiple write commands to disk |
| no | Let the OS decide when it should sync |
It is best to use everysec, which can not only avoid the performance impact caused by writing every time, but also avoid the possible loss of a certain amount of data caused by the crash of the operating system. Even if the system crashes, the user will only lose the data generated within one second at most. Data, when the hard disk is busy performing write operations, Redis will gracefully slow down its speed to accommodate the maximum write speed of the hard disk.
Disadvantages: Because Redis will continuously record the executed write commands into the AOF file, the volume of the AOF file will continue to grow as Redis continues to execute. Under extreme conditions, AOF may even use up all the available space on the hard disk. .
In order to solve the above shortcomings, Redis provides the BGREWRITEAOF command, which rewrites the AOF file by removing redundant commands in the AOF file, making the AOF file as small as possible. Its principle is similar to the BGSAVE command. Redis will create a child process, and then the child process is responsible for rewriting the AOF file. Because the rewriting of the AOF file also requires the child process, there is also snapshot persistence due to the creation of the child process. performance issues and memory usage issues.