Redis persistence: RDB and configuration and Comparative AOF
1. Introduction
Redis performance is due to its all data stored in the memory, in order to make Redis after the restart still ensure data is not lost,
Need to synchronize data from memory to the hard drive, this process is persistence.
Redis supports persistent in two ways, one is RDB ways, one way is to AOF. Which may be used alone or both in combination.
2. RDB persistence
persistence mechanism: within the specified time interval the memory snapshot of the data set to be written to disk
RDB persistent way is through snapshots (snapshotting) completed, Redis will automatically take a snapshot of the data in memory when certain conditions are met and persisted to disk.
Conditions snapshot may be customized by the user profile, constituted by two parameters: the number of key changes and time.
When the key number is changed within a specified period of time greater than the specified value will snapshot.
RDB is a persistent manner Redis used by default, the configuration file has been preset three conditions:
at least one key is changed within save 900 1 # 900 seconds to a snapshot
There are at least 10 keys are changed within save 300 10 # 300 seconds to a snapshot
There are at least 10,000 key changes within save 60 10000 # 60 seconds to take a snapshot
There may be multiple conditions, it is "or" relationship between the conditions, as long as a condition is met, it will take a snapshot.
If you want to disable automatic snapshots, just to save all the parameters can be deleted.
Redis will default snapshot file is stored in the current directory (may CONFIG GET dir to see) the dump.rdb file, you can specify the storage path and file name of the snapshot file by configuring dir and dbfilename two parameters.
RDB persistence configuration (modified redis.conf file)
RDB persistent configuration (modified redis.conf file) NOTE: Standard Red profile is modified portion
deamonize start the program in the form of a process of change: yes
timeout client does not interrupt the operation of the service without interruption 0
RDB configuration snapshot
database default how many libraries 16 libraries
save 9001 every 900 seconds there is a snapshot of a single operation
rdbcompression compressed snapshots yes
dbfilename where you can modify the name dump.rdb
dir ./ generated by default the current directory (the directory can be modified)
File repair: redis-check-dump (what a sudden power outage caused by file corruption)
without losing the snapshot data through shutdwon, directly kill -9 does not directly kill the abnormal shutdown backup.
RDB process snapshots of
i.Redis using the fork function copy of the current process (the parent) a copy of (the child);
. Ii parent process continues to receive and process commands sent by the client, and the child will begin the process of memory data is written to a temporary file on your hard disk;
. iii when the child process all the data written to the RDB will replace the old file with the temporary file, bringing a snapshot operation is complete.
Replication at the time the fork of the operating system (Unix-like operating systems) will be used to write (copy-on-write) strategy,
the moment the parent and child that is fork function happens to share the same memory data, when the parent you want to change where a slice data (such as executing a write command),
the operating system will copy data sheet to ensure that the data the child process is not affected, so the new RDB file storage is a fork moment of memory data.
Redis during the snapshot process does not modify RDB file, only after the end of the snapshot will replace the old file into the new, that is to say at any time RDB files are complete.
This allows us to achieve Redis database backups through the regular backup RDB files.
RDB file is compressed (rdbcompression parameters can be configured to disable compression to save CPU usage) binary format, so the space is often smaller than the size of the data in memory, more conducive to transmission.
In addition to automatic snapshot, you can also manually send SAVE or BGSAVE Redis commands let snapshot,
the difference between the two commands is that, SAVE snapshot is operated by the main process, will block other requests to live, to replicate a child BGSAVE will be executed by the fork function redis snapshot process to operate.
After the RDB Redis starts, it reads the snapshot file, the data is loaded from the hard disk into memory.
The amount of data with different structures and server performance, this time is different. Typically ten million records a character string type of the keys, the size of 1GB snapshot file is loaded into memory takes 20 to 30 seconds.
For persistence by RDB way, once Redis abnormal exit, you will lose all data changes after the last snapshot.
This requires the developer according to the particular application, the conditions set by the automatic snapshot data combination to the possible loss of control in the acceptable range.
If the data is very important that they can not afford any losses, you might consider using AOF way for persistence.
RDB persistence advantages and disadvantages
advantages: the entire Redis database will contain only one file, which for file backup is perfect
Disadvantage: rdb based on data from time to time to conduct a snapshot of persistent, if the event of a server failure at this time, will be disastrous.
Resolution: This requires the developer according to the particular application, the conditions set by the automatic snapshot data combination to the possible loss of control in the acceptable range.
If the data is very important that they can not afford any losses, you might consider using AOF way for persistence.
3. AOF persistence
persistence mechanism: each record server operation, at the beginning of Redis server starts, it reads this file to rebuild the database to ensure data after starting the database is intact form of logs.
AOF persistence configuration (modify redis.conf file)
by default Redis is not turned AOF (append only file) persistence mode can be turned on by appendonly parameters redis.conf in:
appendonly yes
Redis will be executed one by one AOF file at boot hard disk data to be loaded into memory, load speed will be slower compared with RDB
open every command execution data Redis a change in the persistence of the AOF , Redis command will write the AOF file on your hard disk.
AOF same location where you saved the file and RDB files are set by the dir argument, the default file name is appendonly.aof, can be modified by appendfilename parameters:
appendfilename appendonly.aof
Redis automatic configuration file AOF rewrite conditions
auto-aof-rewrite-percentage 100 # AOF when what percentage of file size when more than last AOF rewrite current file size will be rewritten again, if you have not weight wrote, AOF file size based on the places to start
auto-aof-rewrite-min-size 64mb # AOF allows rewriting the minimum file size
AOF file manually override
Bgrewriteaof performed manually rewrite
Rewriting process data in memory only and is related to and unrelated to the previous aof file.
The so-called "rewrite" is actually a term ambiguous, in fact, does not need to rewrite AOF AOF original file any written and read,
It is aimed at current values in the database keys.
After the configuration file is written AOF, it requires the system to refresh the disk cache mechanism
# Appendfsync always # write each execution would perform a synchronization, the safest and most slow
appendfsync everysec # execute every second synchronization
# Appendfsync no # do not take the initiative to synchronize, but entirely up to the operating system to do (ie, every 30 seconds), the fastest and most insecure
File Repair
redis-check-aof
Redis allows simultaneous open AOF and RDB, both to ensure data security and backup, etc. so that the operation is very easy. At this time restart Redis Redis will use the AOF files to recover data, because data may be lost persistent way less AOF
AOF persistent merits of the
AOF persistent manner the advantages and disadvantages:
Advantages: Redis provides synchronization strategy 3, that is synchronized every second, every modification synchronous and non-synchronous
1) to modify the sync: synchronize once every modification time, the performance of consumption is very large.
2) second synchronization: the operation performed once per second synchronization
3) are not synchronized
Disadvantages: For the same amount of data sets, AOF files are usually larger than the RDB file, and depending on the synchronization policy, AOF tend to slow in the operating efficiency in RDB
4. Dynamic persistence scheme switching
dynamically switched redis lasting manner, to switch from the AOF of RDB (Redis2.2 support and above)
the CONFIG the SET appendOnly Yes
CONFIG SET save ""（可选）
Note: When redis start, if aof rdb persistence and persistence are open, the program will give priority to the use aof way to restore the data set,
because aof the way the saved data is usually the most complete. If aof file is missing, then after starting the database is empty.
Note: If you want to redis database running, switching from RDB to AOF, it is recommended to use the dynamic switching mode, and then modify the configuration file, restart the database.
(Not directly modify the configuration file, restart a database, or the data in the database is empty.)
5. persistent recovering data server is down (from the master copy)
by a persistence function, to ensure that even the Redis server restart case is not the loss (or a small loss) data.
However, because the data is stored on a server if that server hard disk failure, can lead to data loss.
In order to avoid a single point of failure, we want to copy multiple copies of the database to be deployed on different servers, even if one server fails the other server can still continue to provide services.
This requires that when a database is updated on the server, can automatically update the data synchronization to other servers, the Redis provided a copy (Replication) process synchronization function automatically.
Configuration
was added slaveof master-ip master-port from the profile database through a configuration file, the configuration without the primary database
By the time the command-line parameters to start redis-server using command line parameters --slaveof master-ip master port
redis-server --port 6380--slaveof 127.0.0.16379
Command SLAVEOF master-ip master-port
redis>SLAVEOF 127.0.0.16379
SLAVEOF NO ONE stop receiving the current database can be synchronized to other databases, conversion to the primary database
Main advantages of the copy application scenarios and
isolation 1. The reader may be implemented separate read and write to improve the load capacity of the server by copying.
In a typical scenario, a reading greater than the frequency of writing, when the stand-alone Redis can not cope with a large number of read requests (especially the request for a more resource intensive, such as SORT command, etc.) can be established from multiple databases through replication,
Master database only write, and read from the database responsible.
2. relatively time consuming typically lasting from a persistent database, in order to improve performance, can create a (or several) from the database by copying function, and enable persistence, while disabling the persistent database from the master database.
When the crash from the database will automatically restart the emperor database data synchronization over, so no need to worry about data loss.
When the primary database crashes need to upgrade to continue to serve in the master database to use SLAVEOF NO ONE command from the database from the database,
And use the command to SLAVEOF provided from the database, the data can be synchronized back to the new primary database after the original primary database started.
6. Other relevant config command
redis the config command
Use config set to dynamically set the parameter information after a server restart becomes ineffective.
config set appendonly yes
config set save "90 1 30 10 60 100
Use config get to see all the parameters can be set using the command config set
config get *
Use config rewrite command when you start Redis server specified redis.conf file is rewritten (Redis2.8 and above can use),
mainly to use config set dynamically assigned command to save the configuration file.
config rewrite
Note: config rewrite command for rewriting redis.conf file is atomic, and is the same:
If you override the error during a server crash or rewrite, rewrite it fails, the original file is not modified redis.conf.
If the rewrite is successful, redis.conf file as a new file rewritten.
ps -ef | grep redis view the process service is started
kill the process kill -93004
bgrewriteaof manually override aof
requirepass password access password configuration redis
auth password Client login access code entered
