Outline of this article:
-
Introduction of two persistence mechanisms: RDB and AOF
-
Advantages of RDB persistence mechanism
-
Disadvantages of RDB persistence mechanism
-
Advantages of AOF persistence mechanism
-
Disadvantages of AOF persistence mechanism
-
How to choose between RDB and AOF
We already know that for an enterprise-level redis architecture, persistence is irreducible
Enterprise-level redis cluster architecture:
Mass data, high concurrency, high availability
Persistence is mainly used for disaster recovery and data recovery. It can also be classified as a high-availability link. For example, if your redis is completely hung up, then redis is unavailable. What you have to do is to make redis available. Become available as soon as possible, restart redis, and let it provide external services as soon as possible, but if you have not done data backup, redis is started at this time, it is not available, the data is gone, it is possible to say that a large number of requests come, cache all Unable to hit, no data can be found in redis, it is dead at this time, the cache avalanche problem, all requests, if not hit in redis, will go to the data source of mysql database to find, and mysql will take on high concurrency all at once , And then it hangs up, mysql hangs up, you can't find the data to restore to redis, where does the redis data come from? From mysql.
If you do a good job of redis persistence, and the backup and recovery plan is at an enterprise level, then even if your redis fails, you can quickly restore it by backing up data. Once restored, provide services immediately.
Redis persistence is related to high availability, which is explained in the enterprise redis architecture
Redis persistence:
-
RDB
-
AOF
1. Introduction of two persistence mechanisms: RDB and AOF
RDB persistence mechanism to perform periodic persistence of data in redis
The AOF mechanism uses each write command as a log and writes it to a log file in append-only mode. When redis restarts, the entire data set can be rebuilt by replaying the write command in the AOF log.
If we want redis to be used only as a pure memory cache, then all the persistence mechanisms of RDB and AOF can be prohibited
Through RDB or AOF, the data in redis memory can be persisted to disk, and then these data can be backed up to other places, such as Alibaba Cloud, cloud services
If redis hangs, the memory on the server and the data on the disk are lost. You can copy back the previous data from the cloud service, put it in the specified directory, and then restart redis, redis will automatically persist the data files according to To restore the data in the memory and continue to provide external services
If both RDB and AOF persistence mechanisms are used at the same time, when redis restarts, AOF will be used to reconstruct the data, because the data in AOF is more complete.
2. The advantages of RDB persistence mechanism
(1) RDB will generate multiple data files, each of which represents the data of redis at a certain moment. This multiple data file method is very suitable for cold backup, and this complete data file can be sent Go to some remote secure storage, such as Amazon's S3 cloud service, which can be Alibaba Cloud's ODPS distributed storage in China, and regularly back up the data in redis with a predetermined backup strategy
(2) RDB has very little impact on the external read and write services provided by redis. Redis can maintain high performance, because the main redis process only needs to fork a child process, and let the child process perform disk IO operations for RDB persistence.
(3) Compared with the AOF persistence mechanism, it is faster to restart and restore the redis process directly based on the RDB data file.
3. Disadvantages of RDB persistence mechanism
(1) If you want to lose as little data as possible when redis fails, RDB is not as good as AOF. Generally speaking, RDB data snapshot files are generated every 5 minutes or longer. At this time, it is necessary to accept that once the redis process goes down, the last 5 minutes of data will be lost.
(2) Each time RDB executes RDB snapshot data file generation in the fork subprocess, if the data file is particularly large, it may cause the service provided by the client to be suspended for several milliseconds, or even several seconds.
4. The advantages of AOF persistence mechanism
(1) AOF can better protect data from loss. Generally, AOF will perform an fsync operation through a background thread every 1 second, and lose up to 1 second of data
(2) The AOF log file is written in append-only mode, so there is no disk addressing overhead, the writing performance is very high, and the file is not easy to be damaged, even if the file tail is damaged, it is easy to repair
(3) Even if the AOF log file is too large, the background rewrite operation will not affect the client's reading and writing. Because when rewrite log, the instructions in it will be compressed to create a minimum log that needs to recover data. When creating a new log file, the old log file is still written as usual. When the new merged log file is ready, exchange the old and new log files.
(4) The commands of the AOF log file are recorded in a very readable way. This feature is very suitable for emergency recovery of catastrophic accidental deletion. For example, if someone accidentally emptied all the data with the fluxhall command, as long as the background rewrite has not occurred at this time, then you can copy the AOF file immediately, delete the last fluxhall command, and then put the AOF file back. Through the recovery mechanism, all data is automatically recovered.
5. Disadvantages of AOF persistence mechanism
(1) For the same piece of data, the AOF log file is usually larger than the RDB data snapshot file
title
(2) After AOF is turned on, the supported write QPS will be lower than the write QPS supported by RDB, because AOF is generally configured to fsync log files once per second, of course, once fsync per second, the performance is still very high
(3) There have been bugs in AOF before, that is, when data is restored through the logs recorded by AOF, the exact same data is not restored. Therefore, a more complex command log/merge/playback method like AOF is more fragile than a method based on RDB that persists a complete data snapshot file each time and is prone to bugs. However, AOF is to avoid bugs caused by the rewrite process, so each rewrite is not based on the old instruction log for merge, but based on the data in the memory at the time to rebuild the instruction, so the robustness will be much better.
6. How to choose between RDB and AOF
(1) Don't just use RDB, because that will cause you to lose a lot of data
(2) Don't just use AOF, because that has two problems. First, if you use AOF for cold backup, without RDB for cold backup, the recovery speed will be faster; second, RDB simply generates data snapshots every time. , More robust, can avoid the bug of the complex backup and recovery mechanism of AOF
(3) Comprehensive use of AOF and RDB two persistence mechanisms, AOF is used to ensure that data is not lost, as the first choice for data recovery; RDB is used for different degrees of cold backup, when AOF files are lost or damaged and unusable At that time, you can also use RDB for fast data recovery.