Microservice Study Notes--Advanced--(Redis Persistence)

distributed cache

Redis cluster

The problem with single-point Redis

Data loss problem: Redis memory storage, service restart may lose data

Concurrency problem: Although the concurrency capability of single-node Redis is good, it cannot meet high concurrency scenarios such as 618

Fault recovery problem: If Redis goes down, the service is unavailable, and an automatic fault recovery method is needed

Storage capacity problem: Redis is based on memory, and the amount of data that can be stored by a single node is difficult to meet the needs of massive data

Solution:
data loss problem: realize Redis data persistence

Concurrency issues: Build a master-slave cluster to achieve read-write separation

Failure Recovery Issues: Using Redis Sentinels to Realize Health Detection and Automatic Recovery

Storage capacity problem: build a fragmented cluster and use the slot mechanism to achieve dynamic expansion

Redis persistence

RDB persistence
AOF persistence

RDB

The full name of RDB is Redis Database Backup file (Redis data backup file), also known as Redis data snapshot. Simply put, all data in memory is recorded to disk. When the Redis instance fails and restarts, read the snapshot file from the disk and restore the data.

Snapshot files are called RDB files, which are saved in the current running directory by default.

redis-cli  #连接redis客户端
sava   #由Redis主进程来执行RDB，会阻塞所有命令
bgsave  #推荐使用这个命令，开启子进程执行RDB，避免主进程受到影响

RDB will be automatically executed once when Redis is down.

First, you need to install the Redis installation document in the Linux system
:

First, you need to install the dependencies required by REdis:

yum install -y gcc tcl

Then upload the redis installation package in the data to any directory of the virtual machine, eg: /temp directory

unzip:

tar -xvf redis-6.2.4.tar.gz

Enter the redis directory:

cd redis-6.2.4

Run the compile command:

make && make install

If there are no errors, the installation should be successful.

Then modify some configurations in the redis.conf file

#绑定地址  默认是127.0.0.1 会导致只能在本地访问。修改为0.0.0.0则可以在任意Ip访问
bind 0.0.0.0
# 数据库数量，设置为1
databases 1

Start Redis:

redis-sever redis.conf

Stop the redis service:

redis-cli shutdown

Redis has a mechanism to trigger RDB, which can be found in the redis.conf file, and the format is as follows:

#900秒内，如果至少有1个key被修改，则执行bgsave，如果是save ""则表示禁用RDB
save  900  1
save  300  10
save  60  10000

Other configurations of RDB can also be set in the redis.conf file:

# 是否压缩，建议不开启，压缩也会消耗cpu，磁盘的话不值钱
rdbcompression yes

# RDB文件名称
dbfilename dump.rdb

# 文件保存的路径目录
dir ./

At the beginning of bgsave, the main process will be forked to obtain the child process, and the child process will share the memory data of the main process. After the fork is completed, the memory data is read and written to the RDB file.

fork uses copy-on-write technology:

When the main process performs a read operation, it accesses the shared memory;
When the main process performs a write operation, a copy of the data is copied and the write operation is performed.

summary:

Basic process of bgsave in RDB mode?

Fork the main process to get a child process, shared memory space
The child process reads memory data and writes it to a new RDB file
Replace old RDB files with new RDB files

When will RDB be executed? What does save 60 1000 mean?

The default is when the service is stopped
It means that RDB is triggered when at least 1000 modifications are performed in 60 seconds

Disadvantages of RDB?

The RDB execution interval is long, and there is a risk of data loss between two RDB writes.
Fork subprocesses, compression, and writing out RDB files are time-consuming

AOF

The full name of AOF is Append Only File (append file). Every write command processed by Redis will be recorded in the AOF file, which can be regarded as a command log file.

AOF is disabled by default, you need to modify the redis.conf configuration file to enable AOF:

# 是否开启AOF功能，默认是no
appendonly yes
# AOF文件的名称
appendfilename "appendonly.aof"

The frequency of AOF command recording can also be configured through the redis.conf file:

# 表示每执行一次写命令，立即记录到AOF文件
appendfsync always
# 写命令执行完先放入AOF缓冲区，然后表示每隔1秒将缓冲区数据写到AOF文件，是默认方案
appendfsync everysec
# 写命令执行完先放入AOF缓冲区，由操作系统决定何时将缓冲区内容写回磁盘
appendfsync no

configuration item	Timing of brushing	advantage	shortcoming
Always	Swipe screen synchronously	High reliability, almost no data loss	performance impact
everysec	Brush per second	moderate performance	Up to 1 second of data loss
no	operating system control	best performance	Poor reliability, possible loss of large amounts of data

Because it is a record command, the AOF file will be much larger than the RDB file. Moreover, AOF will record multiple write operations to the same key, but only the last write operation is meaningful. By executing the bgrewriteaof command, the AOF file can be rewritten to achieve the same effect with the fewest commands.

Redis will also automatically rewrite the AOF file when the threshold is triggered. Thresholds can also be configured in redis.conf:

# AOF文件比上次文件 增长超过多少百分比则触发重写
auto-aof-rewite-percentage 100
# AOF文件体积最小多大以上才触发重写
auto-aof-rewrite-min-size 64mb

RDB and AOF each have their own advantages and disadvantages. If the data security requirements are high, they are often used in combination in actual development.

	RDB	AOF
Persistence	Take snapshots of the entire memory at regular intervals	Log every command executed
data integrity	Incomplete, lost between backups	Relatively complete, depending on the brushing strategy
File size	There will be compression, the file size is small	Record commands, the file size is very large
Downtime recovery speed	soon	slow
Data Recovery Priority	Low because data integrity is not as good as AOF	High because of higher data integrity
System resource usage	High, heavy CPU and memory consumption	Low, mainly disk IO resources, but AOF takes up a lot of CPU and memory resources when rewriting
scenes to be used	Data loss can be tolerated for several minutes, and the pursuit of faster startup speed	Higher data security requirements are common