In addition to query operations in the database, the three operations of Insert, Delete and Update will affect the data, because transaction processing can ensure that a series of operations can be completely executed or not executed at all, so in After a transaction is committed, an undo log (Undo Log) will be generated when any SQL statement in the transaction is executed.

1.2.2 Redis transactions

The redis transaction provides a mechanism of "packaging multiple commands and then executing them one-time and sequentially", and the transaction will not be actively interrupted during the execution - the server will execute all the commands in the transaction. Continue processing other commands from other clients.

A transaction in Redis will go through three stages from the beginning to the execution: starting the transaction (muiti) , command enqueuing and executing the transaction (exec) . The commands in the transaction are not executed when they are added, and will not be executed until they are submitted (Exec ) in one go.

1.2.2.1 Two different error handling methods

1) Code syntax error (compile-time exception)

When a code syntax error occurs, none of the commands are executed.

2) Code logic error (runtime error)

When a code logic error occurs, other commands can be executed normally (this point does not guarantee the atomicity of the transaction)

Why does redis not support rollback to guarantee atomicity

1.2.2.2 Advantages of this approach:

Redis commands can only fail because of incorrect syntax (and these problems cannot be detected when enqueuing), or because the command is used on the wrong type of key: that is, from a practical point of view, the command that fails It is caused by programming errors, and these errors should be found in the development process, and should not appear in the production environment.
Since there is no need to support rollbacks, the internals of Redis can be kept simple and fast.

Since there is no mechanism to avoid errors caused by programmers themselves, and such errors usually do not appear in a production environment, Redis chooses a simpler and faster way to handle transactions without rollback.

1.3 Transaction Monitoring

Redis uses watch key to monitor specified data, which is equivalent to adding optimistic lock

Watch ensures that transactions can only be executed on the premise that all monitored keys have not been modified. If this premise cannot be met, the transaction will not be executed.

Watch execution process:

2. Data persistence

Redis is an in-memory database. Once the server process exits, the data in the database will be lost. In order to solve this problem, Redis provides two persistence solutions, which save the data in memory to disk to avoid data loss. Persistence methods: snapshot (RDB file) and append file (AOF file) . The principles of the two methods are introduced below.

The RDB persistence method will save the data snapshot at that point in time at a specific interval.
The AOF persistence method will record the write operations received by each server. When the service starts, these recorded operations will be executed one by one to reconstruct the original data. The format of the write operation command record is consistent with the Redis protocol, and it is saved in an appended manner.
The persistence of Redis can be disabled, which means that you can make the life cycle of the data only exist during the running time of the server.
The persistence of the two methods can exist at the same time, but when Redis restarts, the AOF file will be prioritized for rebuilding the data.

2.1 RDB persistence

The file generated by RDB persistence is a compressed binary file. This file can be saved to the hard disk, and the state of the database can be restored through this file. It can be executed manually, or configured in the redis.conf configuration file, and executed regularly .

2.1.1 Working principle

When performing RDB, the main process of redis will not do io operation, but will fork a sub-process to complete the operation:

1) Redis calls forks. Have both parent and child processes.

2) The child process writes the data set into a temporary RDB file.

3) When the child process finishes writing the new RDB file, Redis replaces the original RDB file with the new RDB file and deletes the old RDB file.

This way of working allows Redis to benefit from the copy-on-write mechanism (because the child process is used for write operations, and the parent process can still receive requests from clients)

2.1.2 Trigger mechanism

There are two types of triggers for RDB persistence in Redis: manual triggers and automatic triggers.

2.1.2.1 Manual Trigger

1）save

The save command is a synchronous command, which will occupy the main process, cause blocking, and block all client requests

2) bgsave

bgsave is performed asynchronously. When persisting, redis can continue to respond to client requests

Comparison between bgsave and save:

Order	save	bgsave
I/O type	Synchronize	asynchronous
block	yes	Yes (blocking happens at fock(), usually very fast)
the complexity	O(n)	O(n)
advantage	Does not consume additional memory	Does not block client commands
shortcoming	Blocking client commands	Requires fock child process, consumes memory

2.1.2.2 Automatic Trigger

Triggering conditions:

Save automatically triggers the configuration, see the configuration below, satisfying that the key can be modified n times within m seconds, triggering rdb

# 时间策略   save m n m秒内修改n次key，触发rdb
save 900 1
save 300 10
save 60 10000

# 文件名称
dbfilename dump.rdb

# 文件保存路径
dir /home/work/app/redis/data/

# 如果持久化出错，主进程是否停止写入
stop-writes-on-bgsave-error yes

# 是否压缩
rdbcompression yes

# 导入时是否检查
rdbchecksum yes

1) When the slave node is fully copied, the master node sends the rdb file to the slave node to complete the copy operation, and the master node will trigger the bgsave command;

2) Execute the flushall command, which will trigger rdb

3) Exit redis and aof is not enabled

2.1.3 Advantages and disadvantages of RDB

2.1.3.1 Advantages:

1) The content of RDB is binary data, which occupies less memory, is more compact, and is more suitable as a backup file;

2) RDB is very useful for disaster recovery. It is a compact file that can be transferred to a remote server for faster Redis service recovery;

3) RDB can improve the running speed of Redis to a greater extent, because the Redis main process will fork() a child process every time it is persisted to persist data to disk, and the Redis main process will not perform disk I/O, etc. operate;

4) Compared with AOF format files, RDB files can be restarted faster.

2.1.3.2 Disadvantages:

1) Because RDB can only save data for a certain time interval, if the Redis service is terminated unexpectedly midway, the Redis data for a period of time will be lost.

2) RDB requires frequent fork() to persist it on disk using subprocesses. Fork() can be time-consuming if the dataset is large, and can cause Redis to stop serving clients for a few milliseconds or even a second if the dataset is large and the CPU performance is poor.

2.2 AOF persistence

Record each write operation in the form of a log, record all instructions executed by Redis (read operations are not recorded), only append files but not rewrite files, redis will read the file and rebuild data at the beginning of startup , In other words, if redis restarts, the write command will be executed from front to back according to the content of the log file to complete the data recovery work.

2.2.1 AOF configuration items

# 默认不开启aof  而是使用rdb的方式
appendonly no

# 默认文件名
appendfilename "appendonly.aof"

# 每次修改都会sync 消耗性能
# appendfsync always
# 每秒执行一次 sync 可能会丢失这一秒的数据
appendfsync everysec
# 不执行 sync ,这时候操作系统自己同步数据，速度最快
# appendfsync no

The whole process of AOF can be roughly divided into two steps. The first step is to write commands in real time (if it is configured with appendfsync everysec, there will be 1s loss), and the second step is to rewrite the aof file.

2.2.2 AOF rewriting mechanism

With the operation of Redis, the AOF log will become longer and longer. If the instance is down and restarted, it will be very time-consuming to replay the entire AOF. In the log records, there are many meaningless records, such as my Now incr a data a thousand times, then there is no need to record these 1000 modifications, only the last value needs to be recorded. So AOF rewriting is required.

Redis provides the bgrewriteaof command to rewrite the AOF log. When the command runs, it will open a sub-process to traverse the memory, and then convert it into a series of Redis operation instructions, and then serialize it into a log file. After completion, replace the original AOF file, and this is complete.

Similarly, the triggering of the rewriting mechanism can also be configured in redis.config:

By setting no-appendfsync-on-rewrite to yes, the rewrite mechanism is turned on; auto-aof-rewrite-percentage 100 means that the file size has increased by 100% from the last time after writing, and the rewrite is triggered again;

auto-aof-rewrite-min-size 64mb means that when the file reaches at least 64mb, the braking rewrite will be triggered.

2.2.3 Trigger mode

There are two types of AOF persistence triggers in Redis: manual triggering and automatic triggering.

2.2.3.1 Manual Trigger

bgrewriteaof

2.2.3.2 Automatic Trigger

It is triggered according to the configuration rules. Of course, the overall time of automatic triggering is also related to the frequency of Redis's scheduled tasks.

2.2.4 Advantages and disadvantages of AOF

2.2.4.1 Advantages

1) Data security

Aof persistence can be configured with appendfsync attribute, there is always, and every time a command operation is performed, it will be recorded in the aof file once.

2) Solve the problem of data consistency.

Write files through the append mode, even if the server is down in the middle, you can use the redis-check-aof tool to solve the data consistency problem.

3) The rewrite mode of the AOF mechanism.

Before the AOF file is rewritten (commands will be merged and rewritten when the file is too large), some of the commands can be deleted (such as the misoperated flushall))

2.2.4.2 Disadvantages

1) AOF files are larger than RDB files, and the recovery speed is slower.

2) When the data set is large, it is less efficient than rdb startup.

2.3 Comparison between rdb and aof

comparison item	RDB	AOF
boot priority	Low	high
volume	Small	big
recovery speed	quick	slow
data security	lost data	According to the policy decision

2.4 Summary

RDB saves the data at that time point at a specific interval based on the snapshot method. The backup data is relatively small and the recovery speed is fast, but data loss may occur; the AOF persistence method will record the write operations received by each server. The backup file is large in size and the recovery speed is slow, but this method formulates a certain strategy and will not lose data. Relatively speaking, the data security is higher than that of RDB. In a real enterprise production environment, the Redis data backup strategy is generally to enable both RDB and AOF backup methods at the same time. While improving enterprise security as much as possible, data can also be quickly restored based on data snapshots at a certain point in time.

Well, that’s all for this issue of Redis. If you think this article is helpful to you, please like + follow!

Redis transaction and data persistence

1. Redis transaction

1.1 The essence of business

1.2 Database transaction and redis transaction

1.2.1 Database transactions