In web servers, high availability refers to the time when the server can be accessed normally, and the measure is how long the normal service can be provided (99.9%, 99.99%, 99.999%, etc.) .

However, in the context of Redis , the meaning of high availability seems to be broader. In addition to ensuring the provision of normal services (such as master-slave separation, rapid disaster recovery technology), it is also necessary to consider the expansion of data capacity and data security without loss.

In Redis, the technologies to achieve high availability mainly include persistence, replication, sentinels and clusters. The following describes their functions and what problems they solve.

Persistence : Persistence is the simplest high-availability method (sometimes not even classified as a high-availability method). Its main function is data backup, that is, storing data on the hard disk to ensure that data will not be lost due to process exit.

Replication : Replication is the basis of highly available Redis. Sentinels and clusters are based on replication to achieve high availability. Replication mainly implements multi-machine backup of data, as well as load balancing for read operations and simple fault recovery. Defects: Failure recovery cannot be automated; write operations cannot be load-balanced; storage capacity is limited by a single machine.

Sentinel : Based on replication, Sentinel implements automated failure recovery. Defects: Write operations cannot be load-balanced; storage capacity is limited by a single machine.

Cluster : Through clustering, Redis solves the problem that write operations cannot be load-balanced, and the storage capacity is limited by a single machine, and realizes a relatively complete high-availability solution.

1. Redis persistence

Persistence function: Redis is an in-memory database, and data is stored in memory. In order to avoid permanent loss of data caused by process exit, it is necessary to periodically save the data in Redis in some form (data or command) from memory to hard disk; When Redis restarts next time, use the persistent file to achieve data recovery. Additionally, the persistent files can be copied to a remote location for disaster backup purposes.

Redis persistence is divided into RDB persistence and AOF persistence : the former saves the current data to the hard disk, and the latter saves each executed write command to the hard disk (similar to MySQL's binlog); due to the real-time nature of AOF persistence Better, that is, less data is lost when the process exits unexpectedly, so AOF is currently the mainstream persistence method, but RDB persistence still has its place.

1.1 RDB persistence

The full name of RDB is RedisDataBase. RDB persistence is to generate a snapshot of the data in the current process and save it to the hard disk (so it is also called snapshot persistence). The suffix of the saved file is rdb, and the default file name is dump.rdb. When Redis restarts, the snapshot file can be read to restore data.

What is a snapshot? You can understand it as taking a photo of the data at the current moment and saving it.

1.1.1 Trigger conditions

The triggering of RDB persistence is divided into manual triggering and automatic triggering.

1) Manual trigger

Both the save command and the bgsave command can generate RDB files.

The save command will block the current Redis server. During the execution of the save command, Redis cannot process other commands until the RDB process is completed. The specific process is as follows:

When the execution is completed, if there is an old RDB file, replace the old one with the new one. Our clients may be tens of thousands or hundreds of thousands, this method is obviously not advisable.

The bgsave command creates a child process, which is responsible for creating the RDB file, and the parent process (that is, the Redis main process) continues to process requests.

When this command is executed, Redis will perform snapshot operations asynchronously in the background, and snapshots can also respond to client requests. The specific process is as follows:

The specific operation is: the Redis process executes the fork operation to create a sub-process, and the RDB persistence process is in charge of the sub-process, which ends automatically after completion. Blocking only occurs during the fork phase , generally for a short time. Basically all RDB operations inside Redis use the bgsave command

1.1.2 Comparison between save and bgsave commands

During the execution of the bgsave command, only the fork child process will block the server, but for the save command, the entire process will block the server, so save has been basically abandoned, and the use of save should be avoided in the online environment; only the bgsave command will be introduced later . In addition, when automatically triggering RDB persistence, Redis will also choose bgsave instead of save for persistence;

2) Automatic trigger

save m n

The most common case of automatic triggering is to pass save mn in the configuration file to specify that bgsave will be triggered when n changes occur within m seconds. For example, if you view the default configuration file of redis (redis.conf in the root directory of redis under Linux), you can see the following configuration information:

The meaning of save 900 1 is:

When the time reaches 900 seconds, if the redis data has changed at least once, execute bgsave;

The same is true for save 300 10 and save 60 10000.

When any one of the three save conditions is met, bgsave will be called.

1.1.3 Related parameter configuration explanation

The parameters are configured in the redis.conf file.

# save: This is used to configure the persistence conditions that trigger Redis, that is, when to save the data in memory to the hard disk. The default configuration is as follows

save 900 1 #Indicates that if at least one key value changes within 900 seconds, it will be saved

save 300 10 #Indicates that if at least 10 key values change within 300 seconds, save

save 60 10000 #Indicates that if at least 10000 key values change within 60 seconds, save

# The default value is yes. Whether Redis stops receiving data when RDB is enabled and the last background save data fails. This will make the user aware that the data is not properly persisted to disk, otherwise no one will notice that a disaster has happened. If Redis restarts, it can start receiving data again

stop-writes-on-bgsave-error yes

# The default value is yes. For snapshots stored on disk, you can set whether to perform compressed storage. If so, redis will use the LZF algorithm for compression. If you don't want to consume CPU for compression, you can set it to turn off this function, but the snapshot stored on disk will be relatively large.

rdbcompression yes

# The default value is yes. After storing the snapshot, we can also let redis use the CRC64 algorithm for data verification, but this will increase the performance consumption by about 10%. If you want to get the maximum performance improvement, you can turn off this function.

rdbchecksum yes

Set the storage path of the snapshot file. This configuration item must be a directory, not a file name. Use the dbfilename above as the saved filename.

dir /usr/local/var/db/redis/ Set the file name of the snapshot, the default is dump.rdb dbfilename dump.rdb

1.1.4 Advantages and disadvantages of RDB,

advantage: 　

Suitable for large-scale data recovery, if the business does not require high data integrity and consistency, RDB is a good choice.

shortcoming:

RDB snapshot is a full backup, which stores the binary serialized form of memory data, and the storage is very compact. When the snapshot is persisted, a child process will be started to be responsible for the snapshot persistence. The child process will have the memory data of the parent process. The parent process will not reflect the memory modification of the child process, so the data modified during the snapshot persistence will not be reflected. is saved, data may be lost.

1.1.5 RDB backup trigger conditions:

Execute the save or bgsave command to trigger the generation of the dump.rdb file

Executing the shutdown command can trigger the generation of a dump.rdb file

In the configuration file, if the configuration parameter save mn is specified, when n changes occur within m seconds, bgsave will be triggered to generate a dump.rdb file

1.2 AOF persistence

The full name of AOF is Append Only File, which is a log file for redis to record execution instructions.

1.2.1 AOF working principle

Add the write operation to the file. The AOF log is the log after writing. "After writing" means that Redis executes the command first, writes the data into the memory, and then records the log; it records the steps of command execution, which is very In detail, the change process of the data is depicted.

The working mechanism is very simple, redis will append each received write command to the file through the write function. The popular understanding is logging.

1.2.2 AOF persistence principle

Whenever a write command comes, it is directly saved in our AOF file.

1.2.3 AOF backup trigger conditions (configuration file modification of three parameters)

Synchronization for every modification always: Synchronous persistence. Every time a data change occurs, it will be immediately recorded to the disk. The performance is poor but the data integrity is better.

Synchronous everysec every second: asynchronous operation, record every second If there is a downtime within one second, there will be data loss

different no: never sync

Two, RDB backup recovery case

Connect to redis to add and delete data

Execute the save command again

then shut down the process

Move the rdb file away

After restarting, connect the client

You can see that there is no data, because you have moved its default file for storing data

At this time, re-enter the two data. At this time, there is no dump.rdb file in the directory, because the data is stored in the memory at this time. Only by manually executing the save command will the data be written to the dump.rdb file.

If you want the previous data back, turn off the service, and then move the original dump.rdb back

Restart the redis service, connect to the redis client, and query the data; the data is found to be back

Three, AOF backup recovery case

Modify the configuration file

Restart the service, there will be an appendonly.aof file

Connect to the client, check if there is any data, and find that there is no, because we are using AOF at this time, the original data stored using RBD is still there, but we don’t need it;

Reset the data and found that the memory of the appendonly.aof file has changed and is no longer 0

Then shut down the service, remove the appendonly.aof file, start the service again, connect to the client, check the data, and find it is empty;

Then turn off the service, and move back the original aof file,

Check again and find that there is data;