Redis persistence, master-slave backup, comparison with memcache

There are two ways of redis persistence:

 

In layman's terms, data persistence means saving data to disk to ensure that data will not be lost due to factors such as power failure.

Redis needs to frequently synchronize in-memory data to disk to ensure persistence. Redis supports two persistence methods, one is Snapshotting (snapshot) which is also the default method, and the other is Append-only file (abbreviated aof) method. First introduce these two dump methods, and then talk about some of the phenomena and ideas that I have encountered. The previous content is sorted out from the Internet.

Snapshotting
Snapshots are the default persistence method. This method is to write the data in memory to the binary file in the form of a snapshot, and the default file name is dump.rdb. Snapshot persistence can be done automatically through configuration settings. We can configure redis to automatically take snapshots if more than m keys are modified within n seconds. The following is the default snapshot save configuration

save 900 1 #If more than 1 key is modified within 900 seconds, initiate a snapshot save
save 300 10 #300 seconds If more than 10 keys are modified, initiate a snapshot save
save 60 10000

The following describes the detailed snapshot saving process

1. Redis calls fork, and now there is a child process and a parent process.

2. The parent process continues to process client requests, and the child process is responsible for writing the memory content to the temporary file. Due to the copy-on-write mechanism of OS, the parent and child processes will share the same physical page. When the parent process processes the write request, the OS will create a copy of the page to be modified by the parent process instead of writing the shared page. So the data in the address space of the child process is a snapshot of the entire database at the moment of fork.

3. When the subprocess writes the snapshot to the temporary file, replace the original snapshot file with the temporary file, and then the subprocess exits.

The client can also use the save or bgsave command to notify redis to do a snapshot persistence. The save operation saves the snapshot in the main thread. Since redis uses a main thread to process all client requests, this method will block all client requests. So it is not recommended to use. Another point to note is that each snapshot persistence is to completely write the memory data to the disk once, not only the dirty data is incrementally synchronized. If the amount of data is large and there are many write operations, it will inevitably cause a large number of disk IO operations, which may seriously affect performance.

In addition, since the snapshot method is done at a certain interval, if redis is accidentally shut down, all modifications after the last snapshot will be lost. If the application requires that no modifications can be lost, the aof persistence method can be used. Introduce below

Append-only file

aof has better persistence than snapshot mode, because when using aof persistence mode, redis will append each received write command to the file through the write function (the default is appendonly.aof). When redis restarts, it rebuilds the contents of the entire database in memory by re-executing the write commands saved in the file. Of course, since the os will cache the modifications made by the write in the kernel, it may not be written to the disk immediately. In this way, the persistence of the aof method may still lose some modifications. However, we can tell redis through the configuration file when we want to force os to write to disk through the fsync function. There are three ways as follows (default: fsync once per second)

appendonly yes //Enable aof persistence mode
# appendfsync always //Forcibly write to disk every time a write command is received, the slowest, but guarantees complete persistence, appendfsync is not recommended
everysec //Force write every second Enter the disk once, and make a good compromise between performance and persistence. It is recommended to
# appendfsync no // completely dependent on os, the performance is the best, and persistence is not guaranteed

aof 的方式也同时带来了另一个问题。持久化文件会变的越来越大。例如我们调用incr test命令100次,文件中必须保存全部的100条命令,其实有99条都是多余的。因为要恢复数据库的状态其实文件中保存一条set test 100就够了。为了压缩aof的持久化文件。redis提供了bgrewriteaof命令。收到此命令redis将使用与快照类似的方式将内存中的数据 以命令的方式保存到临时文件中,最后替换原来的文件。具体过程如下

1. redis调用fork ,现在有父子两个进程
2. 子进程根据内存中的数据库快照,往临时文件中写入重建数据库状态的命令
3.父进程继续处理client请求,除了把写命令写入到原来的aof文件中。同时把收到的写命令缓存起来。这样就能保证如果子进程重写失败的话并不会出问题。
4.当子进程把快照内容写入已命令方式写到临时文件中后,子进程发信号通知父进程。然后父进程把缓存的写命令也写入到临时文件。
5.现在父进程可以使用临时文件替换老的aof文件,并重命名,后面收到的写命令也开始往新的aof文件中追加。

需要注意到是重写aof文件的操作,并没有读取旧的aof文件,而是将整个内存中的数据库内容用命令的方式重写了一个新的aof文件,这点和快照有点类似。

 

这两种持久化方式有各自的特点,快照相对性能影响不大,但一旦崩溃,数据量丢失较大,而aof数据安全性较高,但性能影响较大,这就得根据业务特点自行选择了。

 

Redis& Memcached:

1  Redis不仅仅支持简单的k/v类型的数据,同时还提供list,set,zset,hash等数据结构的存储。

2  Redis支持数据的备份,即master-slave模式的数据备份。

3  Redis支持数据的持久化,可以将内存中的数据保持在磁盘中,重启的时候可以再次加载进行使用。

 

Redis的主从备份过程:

 

 redis的主从复制策略是通过其持久化的rdb文件来实现的,其过程是先dump出rdb文件,将rdb文件全量传输给slave,然后再将dump后的操作实时同步到slave中

Guess you like

Origin http://43.154.161.224:23101/article/api/json?id=326246795&siteId=291194637