Centos7Redis database persistence, persistence working principle, application scenarios and persistence configuration

1. Database overview

Redis is an advanced key-value database. It is similar to memcached, but the data can be persisted, and the supported data types are very rich. There are strings, linked lists, sets and ordered combinations. It supports the calculation of set union and complement (difference) on the server side, and also supports multiple sorting functions. So Redis can also be called a data structure server.

Second, the role of persistence and its implementation

1. Function

All data of Redis is stored in memory. If there is no configuration persistence, all data will be lost after redis restart. So you need to turn on the redis persistence function and save the data on the disk. When redis restarts, you can download To restore data. Then save it to disk asynchronously from time to time (semi-persistent mode); you can also write every data change to an append only file (full persistent mode). If two persistence methods are enabled in the server, the AOF persistence method is executed by default;

2. Implementation

• RDB Persistence: Dump Redis's data records in memory to disk at regular intervals, which is similar to the snapshot function.
• AOF persistence: append only file-The principle is to write the Redis operation log to the file in an appended manner, which is approximately real-time.

3. The difference between the two

1. RDB persistence refers to writing a snapshot of a data set in memory to disk within a specified time interval. The actual operation process is to fork a child process. The data set is first written to a temporary file. After the write is successful, the previous file is replaced. , Use binary compression storage.
   
2. AOF persistence is to record every write and delete operation processed by the server in the form of a log. The query operation will not be recorded. It is recorded in the form of a file. You can open the file to see the detailed operation record.
   

1) Advantages and disadvantages of RDB

Advantage:

1. Once this method is adopted, your entire Redis database will only contain one file, which is perfect for file backup. For example, you can plan to archive the last 24 hours of data once every hour, and also archive the last 30 days of data once a day. Through such a backup strategy, once the system has a catastrophic failure, we can recover very easily.
2. For disaster recovery, RDB is a very good choice. Because we can easily compress a single file and transfer it to other storage media.
3. Maximize performance. For the Redis service process, when it starts to persist, the only thing it needs to do is to fork the child process, and then the child process will complete the persistence work, so that it can greatly avoid the service main process to perform IO operations .
4. Compared with the AOF mechanism, if the data set is large, the RDB startup efficiency will be higher.

Disadvantages:

1. If you want to ensure high data availability, that is, to avoid data loss to the maximum extent, then RDB will not be a good choice. Because once the system is down during the timed persistence period, the data that has not had time to write to the disk before will be lost.
2. Since RDB assists in the completion of data persistence through the fork sub-process, if the data set is large, it can cause the entire server to stop serving hundreds of milliseconds, or even 1 minute.

2) Advantages and disadvantages of AOF

Advantage:

1. This mechanism can bring higher data security, that is, data durability. Redis provides 3 synchronization strategies, namely, synchronization per second, synchronization per modification, and asynchronous. In fact, synchronization per second is also completed asynchronously, and its efficiency is also very high. The difference is that once the system is down, the modified data within this second will be lost. And every time the synchronization is modified, we can regard it as synchronization persistence, that is, every data change that occurs will be immediately recorded to the disk. It can be predicted that this method is the lowest in efficiency.
2. Because this mechanism uses append mode for the write operation of the log file, even if there is a downtime during the writing process, the existing content of the log file will not be destroyed. However, if we only write half of the data in this operation, the system crashes, don't worry, we can use the redis-check-aof tool to help us solve the data consistency problem before Redis is started next time.
3. If the log is too large, Redis can automatically enable the rewrite mechanism. That is, Redis continuously writes the modified data to the old disk file in append mode, and at the same time, Redis will also create a new file to record which modification commands have been executed during this time. Therefore, data security can be better guaranteed during rewrite switching.
4. AOF contains a clearly formatted, easy-to-understand log file to record all modification operations. In fact, we can also complete the data reconstruction through this file.
   Disadvantages:
5. For the same number of data sets, AOF files are usually larger than RDB files. RDB is faster than AOF when recovering large data sets.
6. According to different synchronization strategies, if the amount of data is relatively large, AOF is often slower than RDB in terms of operating efficiency.

3) Application scenarios

1. Willing to sacrifice some performance (choose AOF);
2. In exchange for higher cache consistency, to ensure the highest possible integrity of the database (choose AOF);
3. When you are willing to write frequently, do not enable backup in exchange for higher performance. When you manually run save, do a backup (RDB);

Three, RDB persistence mode configuration

1. Install Redis service

Need to upload Redis software:
https://pan.baidu.com/s/1JlhmlSrK1xWerRE5oSgubA
Extraction code: 3gj6

[root@Redis ~]# ls
anaconda-ks.cfg  redis-4.0.9.tar.gz
[root@Redis ~]# tar zxf redis-4.0.9.tar.gz -C /usr/src/
[root@Redis ~]# cd /usr/src/redis-4.0.9/
[root@Redis redis-4.0.9]# make && make PREFIX=/usr/local/redis install
[root@Redis redis-4.0.9]# ln -s /usr/local/redis/bin/* /usr/local/bin/		#创建软连接优化执行路径
[root@Redis redis-4.0.9]# cd utils/
[root@Redis utils]# ./install_server.sh 		#执行安装脚本文件
依次回车即可或自己定义

2. Placement RDB endurance

[root@Redis ~]# vim /etc/redis/6379.conf 
 219 save 900 1			#开启 RDB 快照功能，在 900 秒内完成一个 key 值的变动会触发快照功能
 220 save 300 10		
 221 save 60 10000
 236 stop-writes-on-bgsave-error yes		#当 RDB 快照后台进程失败，不影响用户的写操作
 242 rdbcompression yes						#是否将 RDB 快照文件压缩，关闭后会增加性能
 251 rdbchecksum no							#关闭 RDB 快照文件的检查校验，增加性能
 254 dbfilename dump.rdb					#快照文件的名称
 264 dir /var/lib/redis/6379				#快照文件的存储路径
[root@Redis ~]# /etc/init.d/redis_6379 restart		#重启redis服务
Stopping ...
Redis stopped
Starting Redis server...
[root@Redis ~]# redis-cli 
127.0.0.1:6379> set xingming zhangsan
OK
127.0.0.1:6379> set nianling 23
OK
127.0.0.1:6379> keys *
1) "nianling"
2) "xingming"
127.0.0.1:6379> exit
[root@Redis ~]# reboot  	 #重启验证 key 值是否存在
[root@Redis ~]# redis-cli 
127.0.0.1:6379> keys *
1) "xingming"
2) "nianling"
127.0.0.1:6379> exit

Four, AOF persistence mode configuration

[root@Redis ~]# vim /etc/redis/6379.conf 
将所有 save 配置项都注释掉，表示关闭 RDB 快照功能
 219 #save 900 1
 220 #save 300 10
 221 #save 60 10000
 673 appendonly yes						#开启 AOFF 持久化方式
 677 appendfilename "appendonly.aof"	#指定持久化的文件
 703 appendfsync everysec				#频率，具体参数在下面表格中
 725 no-appendfsync-on-rewrite no		#如若设置为yes，则 redis 执行的命令会存放到缓冲区，待系统自动同步到硬盘
 744 auto-aof-rewrite-percentage 100	#若当前写入的 AOF 文件达到了上次 rewrite 文件大小的 100%，则触发 rewrite 操作          
 745 auto-aof-rewrite-min-size 64mb		#设置 AOF 持久化重写文件的最小值，当达到 60M 并且符合 100% 的条件时，则触发 rewrite 操作          

appendfsync option and synchronization frequency

Options	Sync frequency
always	Every Redis command must be written to disk synchronously. This will seriously reduce Redis performance
everysec	Perform synchronization once per second, and display multiple write commands to the hard disk
no	Let the operating system decide when to synchronize

The AOF file generation process specifically includes three steps: command addition, file writing, and file synchronization.
After Redis turns on the AOF persistence function, after Redis executes a command, it will track the executed write command back to the end of the buffer inside Redis. This process is the appending process of commands. Next, the write command of the buffer will be written to the AOF file. This process is the file writing process. For the operating system, calling the write function does not immediately write the data to the disk. In order to actually write the data to the disk, you also need to call the fsync function, which is a process of file synchronization. Only after the file synchronization process, the AOF file actually saves Redis write commands in the disk.
The appendfsync configuration option is used to configure the frequency of synchronizing the write command to the file. The meaning of the value of each option is shown in the table above. If the value is no, it means that the AOF file is written, but the disk synchronization is not performed. According to the linux system, the disk synchronization is performed by default. The default is 30s, and the efficiency is the highest;

[root@Redis ~]# /etc/init.d/redis_6379 restart		#重启redis服务
Stopping ...
Redis stopped
Starting Redis server...
[root@Redis ~]# redis-cli 
127.0.0.1:6379> keys *
1) "xingming"
2) "nianling"
127.0.0.1:6379> set AJBN AJBM
OK
127.0.0.1:6379> keys *
1) "xingming"
2) "AJBN"
3) "nianling"
[root@Redis ~]# cat /var/lib/redis/6379/appendonly.aof 
*2
$6
SELECT
$1
0
*3
$3
set
$8
xingming
$8
zhangsan
*3
$3
set
$8
nianling
$2
23
*3
$3
set
$4
AJBN
$4
AJBM
[root@Redis ~]# reboot			#重启验证 key 值是否存在
[root@Redis ~]# /etc/init.d/redis_6379 restart		#重启redis服务
Stopping ...
Redis stopped
Starting Redis server...
[root@Redis ~]# redis-cli 
127.0.0.1:6379> keys *
1) "AJBN"
2) "xingming"
3) "nianling"
127.0.0.1:6379> exit