Redis persistent RDB (Redis DataBase), AOF (Append Only File)

Reference: The blogger of station b met the notes written by Mad God. Blogger connection: link .
Redis is a memory database. If the database state in memory is not saved to disk, then once the server process exits, the database state in the server will also disappear. So Redis provides persistence function!

RDB(Redis DataBase)

What is RDB

In master-slave replication, rdb is the backup, on the slave!

The snapshot of the data set in the memory is written to the disk within the specified time interval, that is, the Snapshot in the jargon. When it is restored, the snapshot file is directly read into the memory.

Redis will create (fork) a child process separately for persistence. It will first write the data to a temporary file. After the persistence process is over, it will replace the last persisted file with this temporary file. During the whole process, the main process does not perform any operations. This ensures extremely high performance. If large-scale data recovery is required, and the integrity of the data recovery is not very sensitive, the RDB method is better than the AOF method More efficient. The disadvantage of RDB is that the data may be lost after the last persistence. Our default is RDB, under normal circumstances there is no need to modify this configuration!

The file saved by rdb is dump.rdb, which is configured in the snapshot in our configuration file!

Trigger mechanism

1. When the rules of save are met, the rdb rule will be triggered automatically.
2. Executing the flshall command will also trigger our rdb rule!
3. Exit redis, and an RDB file will also be generated!
Backup will automatically generate a dump.rdb

How to restore rdb files!

1. Just put the rdb file in our redis startup directory, and the dump will be checked automatically when redis starts. rdb restores the data!
2. Check the location that needs to be stored

127.0.0.1:6379> config get dir
1) "dir"
2) "/usr/local/bin"    //如果在这个目录下存在dump。rdb文件，启动就会自动恢复其中的数据

Almost the default configuration of redis is enough, but you still need to understand

Advantages:
1. Suitable for large-scale data recovery! dump.rdb
2, does not require high data integrity!
Disadvantages:
1. It requires a certain time interval for process operation! If redis crashes unexpectedly, the last modified data will be gone!
2. When the process is fork, it will take up a certain amount of memory space!

AOF(Append Only File)

Record all the commands we use, history, and execute all this file when restoring!

What is it

Record each write operation in the form of a log, record all the instructions executed by Redis (read operations are not recorded), only allow additional files but not rewrite files, redis will read the files to rebuild the data at the beginning of the startup. In other words, if redis
restarts, it will execute the write command from front to back according to the content of the log file to complete the data recovery work.

Aof saves the appendonly.aof file

append

appendonly yes    //默认no  不开启状态  yes 开启aof
//重启，redis就可以生效了！

There is an error in the appendonly.aof file. Redis cannot be started. It needs to be repaired. Aof file
redis provides us with a tool. redis-check-aof --fix


If the file is normal, restart it and it can be restored directly!

pros and cons!

appendonly no  //默认是不开启aof模式的，默认使用rdb方式持久化的，在大部分所有的情况下，rdb完全够用！
appendfilename "appendonly.aof"     //持久化的文件的名字

#appendfsync always    //每次修改都会sync。消耗性能
appendfsync everysec    //每秒执行一次，可能会丢失这1s的数据！
#appendfsync no           //不执行sync，这个时候操作系统自己同步数据，速度最快！

#rewrite  //重写

advantage:

1. Every modification is synchronized, and the integrity of the file will be better!
2. Synchronize every second, you may lose one second of data
3. Never sync, the most efficient!

Disadvantages:

1. Equivalent to data files, AOF is much larger than RDB, and the repair speed is also slower than RDB!
2. Aof's operating efficiency is also slower than RDB, so our default configuration for redis is RDB persistence

Rewrite rules

Aof defaults to unlimited appending of files, and the files will get bigger and bigger!

If the aof file is larger than 64m, it is too big! Fork a new process to rewrite the aof file!

Expansion:

1. The RDB persistence method can perform snapshot storage of your data within a specified time interval.
2. The AOF persistence method records each write operation to the server. When the server restarts, these commands will be re-executed to restore the original data , The AOF command uses the Redis protocol to append and save each write operation to the end of the file. Redis can also rewrite the AOF file in the background, so that the volume of the AOF file is not too large.
3. Only cache, if you only want your data to exist when the server is running, you can also not use any persistence
4. Open both persistence methods at the same time

• In this case, when redis restarts, it will load the AOF file first to restore the original data, because under normal circumstances the data set saved by the AOF file is more complete than the data set saved by the RDB file.

• RDB data is not real-time, and when using both at the same time, when the server restarts, only AOF files will be found. Should you just use AOF? The author recommends not, because RDB is more suitable for backing up the database (AOF is not good for backup when it is constantly changing), Restart quickly, and there will be no potential bugs in AOF. Keep it as a just in case.

5. Performance recommendations

• Because the RDB file is only used for backup purposes, it is recommended to only persist the RDB file on the Slave, and it takes only 15 minutes to back up it, and only save the 900 1 rule.
• If you enable AOF, the advantage is that in the worst case, you will only lose no more than two seconds of data. The startup script is simpler and only load your own AOF file. The cost is that it brings a continuous 10, and the second is the end of AOFrewrite. The blockage caused by writing the new data generated in the rewrite process to the new file is almost inevitable. As long as the hard disk is licensed, the frequency of AOFrewrite should be reduced as much as possible. The default value of 64M for AOF rewrite is too small and can be set above 5G. By default, the rewrite can be changed to an appropriate value if it exceeds 100% of the original size.
• If you don't Enable AOF, you can achieve high availability by only relying on Master-Slave Repllcation, which can save a large amount of 10 and reduce the system fluctuations caused by rewrite. The price is that if the Master/Slave is dumped at the same time, more than ten minutes of data will be lost. The startup script must also compare the RDB files in the two Master/Slave and load the newer one. Weibo is this architecture.