Stage 8: Advanced Service Framework (Chapter 3: Distributed Cache Redis)

Distributed cache Redis

– Solve the problems of stand Redis-alone machines based on clusters. There are four major problems of stand-alone machines :Redis
Redis
data loss problem
Redis is an in-memory storage, and data may be lost when the service is restarted.
Concurrency issues
Although the concurrency capability of single-node Redis is good, it cannot meet high concurrency scenarios such as 618.
Recovery issues
If Redis goes down, the service is unavailable and an automatic fault recovery method is required.
Storage capacity issue
Redis is based on memory, and the amount of data that a single node can store cannot meet the demand for massive data.

0.Learning Objectives

1. Redis persistence

Redis has two persistence solutions:

• RDBEndurance
• AOFEndurance

1.1. RDB persistence【important】

  The full name of RDB is Redis Database Backup file (Redis data backup file), also called Redis data snapshot. simply putThat is to record all the data in the memory to the disk . When the Redis instance fails and restarts, the snapshot file is read from the disk and the data is restored.. Snapshot files are called RDB files,The default is to save it in the current running directory of redis。

1.1.1.Execution timing

RDBPersistence is performed in four situations:

• execute savecommand
• execute bgsavecommand
• RedisDuring shutdown
• RDBWhen a condition is triggered

1) save command.
Execute the following command, which can be executed immediately RDB:

The save command will causemain processWhen executing RDB, all other commands will be executed during this process.block. May only be used during data migration .

2) bgsave command
The following command can be executed asynchronously RDB:

after this command is executed, an independent process will be started to complete the RDB.The main process can continue to process user requests without being affected。

3) When shutting down,
Redis a save command will be executed to achieve RDB persistence.

4) Trigger RDB conditions (enabled by default, can be configured by yourself)
There is a mechanism inside Redis to trigger RDB, can redis.confbe found in the file in the following format:

# 900秒内，如果至少有1个key被修改，则执行bgsave ， 如果是save "" 则表示禁用RDB
save 900 1  
save 300 10  
save 60 10000 
save 5 1

RDBOther configurations can also redis.confbe set in the file:

# 是否压缩 ,建议不开启，压缩也会消耗cpu，磁盘的话不值钱
rdbcompression yes

# RDB文件名称
dbfilename dump.rdb  

# 文件保存的路径目录
dir ./ 

1.1.2.RDB principle

  bgsaveAt the beginning, forkthe main process will get the child process.Child processes share the memory data of the main process. After completing the fork, read the memory data and write it to the RDB file.

forkThe technology used is copy-on-write:

• When the main process performs a read operation, the shared memory is accessed;
• When the main process performs a write operation, a copy of the data will be copied and the write operation will be performed.

1.1.3. Summary

What is the basic process of RDB method bgsave?

• Fork the main process to get a child process, sharing memory space
• The child process reads the memory data and writes the new RDB file
• Replace old RDB file with new RDB file

When will RDB be executed? What does save 60 1000 mean?

• save: The default is when the service is stopped
• bgsave: means RDB will be triggered if at least 1000 modifications are performed within 60 seconds.

Disadvantages of RDB?

• The execution interval of RDB is long, and there is a risk of data loss between RDB executions.
• Forking subprocesses, compressing, and writing out RDB files are all time-consuming.

1.2. AOF persistence【important】

1.2.1. AOF principle

  AOFThe full name is Append Only File. RedisEach write command processed will be recorded in the AOF file, which can be viewed ascommand log file（What is recorded inside is the command record executed.）。

1.2.2. AOF configuration

AOF is turned off by default. You need to modify redis.confthe configuration file to turn on AOF:

# 是否开启AOF功能，默认是no
appendonly yes
# AOF文件的名称
appendfilename "appendonly.aof"

Note: If you want to enable it, you need to disable the RDB mentioned above, and save 5 1set it to save ""mean disabledRDB

AOFThe frequency of command recording can also redis.confbe configured through files:

# 表示每执行一次写命令，立即记录到AOF文件
appendfsync always 
# 写命令执行完先放入AOF缓冲区，然后表示每隔1秒将缓冲区数据写到AOF文件，是默认方案
appendfsync everysec 
# 写命令执行完先放入AOF缓冲区，由操作系统决定何时将缓冲区内容写回磁盘
appendfsync no

Most commonly appendfsync everysec , both safety and performance are guaranteed;

Comparison of three strategies:

1.2.3. AOF file rewriting

  Because it is a recording command, the AOF file will be much larger than the RDB file. Moreover, AOF will record multiple write operations on the same key, but only the last write operation is meaningful.By executing bgrewriteaofthe command, you can make the AOF file perform the rewriting function and achieve the same effect with the fewest commands..

  As shown in the figure, AOF originally has three commands, but set num 123 和 set num 666they are all operations on num. The second time will overwrite the first value, so it is meaningless to record the first command.
  So after rewriting the command, the content of the AOF file is:mset name jack num 666

  RedisFiles will also be automatically rewritten when the threshold is triggered AOF. Thresholds can also redis.confbe configured in：

# AOF文件比上次文件 增长超过多少百分比则触发重写
auto-aof-rewrite-percentage 100
# AOF文件体积最小多大以上才触发重写 
auto-aof-rewrite-min-size 64mb 

1.3.Comparison between RDB and AOF

RDBEach has its own advantages and disadvantages. If the requirements for data security are higher, the two are often used in combinationAOF in actual development .

2.Redis master-slave

2.1. Build a master-slave architecture [important】

  The concurrency capability of single-node Redis has an upper limit. To further improve the concurrency capability of Redis, you need toBuild a master-slave cluster to achieve separation of reading and writing.

Please refer to the pre-course materials for the specific construction process."Redis cluster.md":

Detailed content with link

  If some of the above operations are implemented in actual development, there is no need to change the IP, because each master and slave node will use a different host to perform write operations and the slave node to perform read operations
  ;

2.1.1. Summary

2.2. Master-slave data synchronization principle

2.2.1.Full synchronization【important】

When the master-slave establishes a connection for the first time, it will perform full synchronization and mastercopy all the data of the node to slavethe node., Process:

There is a question here, masterhow to know salvethat it is the first time to connect? ?

There are several concepts that can be used as a basis for judgment:

• Replication Id : Replid for short, is the mark of the data set. If the ID is consistent, it means the same data set. Each master has a unique replicad, and the slave will inherit the replicad of the master node.
• offset : offset, which gradually increases as the data recorded in repl_baklog increases. When the slave completes synchronization, it will also record the current synchronization offset. If the slave's offset is smaller than the master's offset, it means that the slave data lags behind the master and needs to be updated.

  Therefore, slavefor data synchronization, you must masterdeclare your own replication id and to determine which data needs to be synchronized offset.master。

  Because the slave is originally a master and has its own replica and offset. When it becomes a slave for the first time and establishes a connection with the master, the replica and offset sent are its own replica and offset.

  The master will send its replica and offset to the slave, and the slave will save this information. From now on, the slave's replica will be consistent with the master's.。

Therefore, the basis for the master to judge whether a node is synchronized for the first time is to see whether the replicaid is consistent .

As shown in the figure:

complete process description:

• slave node requests incremental synchronization
• The master node determines the replica, finds inconsistencies, and rejects incremental synchronization.
• The master generates RDB from the complete memory data and sends the RDB to the slave.
• The slave clears local data and loads the master's RDB.
• The master records the commands during RDB in repl_baklog and continues to send the commands in the log to the slave.
• The slave executes the received command and maintains synchronization with the master.

2.2.2. Incremental synchronization【important】

  Full synchronization needs to be done first RDB, and then the RDB file is transferred to the slave through the network, which is too expensive. Therefore, except for the first full synchronization, the slave and master perform incremental synchronization most of the time .

What is incremental synchronization? That is, only update part of the data that is different slavefrommaster. As shown in the picture:

So masterhow do you know slavethe difference with your own data?

2.2.3. repl_backlog principle

How does the master know the difference between the data of the slave and its own?

This is about the files during full synchronization repl_baklog.

  This file is a fixed-size array, but the array is circular, which means that after the footer reaches the end of the array, it will start reading and writing from 0 again , so that the data at the head of the array will be overwritten.

repl_baklog命令日志will record the and (offset) processed by Redis offset, including the current master offsetand the slave has copied offset: the difference between and is the

  slavedata that needs to be incrementally copied.   As data continues to be written, the master's offset gradually becomes larger, and the slave continues to copy.masteroffsetsalve
Catching up with the master’s offset:

Until the array is filled: At

  this time, if new data is written, the old data in the array will be overwritten. However, as long as the old data is green, it means that it has been synchronized to the slave. Even if it is overwritten, it will have no effect. Because it is only the red part that is not synchronized.

  However, if the slave has network congestion, causing the master's offset to far exceed the slave's offset:

If the master continues to write new data, its offset will overwrite the old data until the slave's current offset is also overwritten: in the

  brown box Red portion,It is data that has not been synchronized but has been overwritten. At this time, if the slave recovers and needs to synchronize, it will find that its offsets are gone and incremental synchronization cannot be completed. Only full sync can be done.

2.3.Master-slave synchronization optimization

Master-slave synchronization can ensure the consistency of master-slave data, which is very important.

RedisYou can optimize the master-slave cluster from the following aspects :

• Enable diskless replication in masterthe configuration (it is better to do it on the network) to avoid disk IO during full synchronization.repl-diskless-sync yes
• The memory usage on a single Redis node should not be too large to reduce excessive disk IO caused by RDB.
• Increase the size appropriately repl_baklogto achieve fault recovery as soon as possible when the slave is found to be down, and avoid full synchronization as much as possible.
• Limit the number of slave nodes on a master. If there are too many slaves, you can use a master-slave-slave chain structure to reduce pressure master.

Master-slave architecture diagram:

2.4. Summary

Briefly describe the difference between full synchronization and incremental synchronization?

• Full synchronization: The master generates RDB from the complete memory data and sends the RDB to the slave. Subsequent commands are recorded in repl_baklogand sent to the slave one by one.
• Incremental synchronization: slave submits its own offset (offset) to the master, and the master obtains the command after the offset in repl_baklog and gives it to the slave

When will full synchronization be performed?

• When the slave node connects to the master node for the first time
• When the slave node has been disconnected for too long and repl_baklogthe offset in has been overwritten

When is incremental synchronization performed?

• When the slave node is disconnected and restored, and repl_baklogthe offset can be found in (the offset master and slave are inconsistent, and the slave node offset is not overwritten)

3.Redis Sentinel

Redis provides the sentinel ( Sentinel) mechanism to implementAutomatic failure recovery of master-slave cluster。

3.1. Sentinel principle

3.1.1. Cluster structure and function

The structure of the sentinel is as follows:

The role of the sentry is as follows：

• Monitoring : Sentinel will continuouslyCheck that your master and slave are working as expected
• Automatic failure recovery : If the master fails, Sentinel willPromote a slave to master. When the failed instance is restored, the new master will take over.
• Notice : Sentinel acts as a service discovery source for Redis clients. WhenWhen a failover occurs in the cluster, the latest information will be pushed to the Redis client.

3.1.2. Cluster monitoring principle

SentinelMonitor the service status based on the heartbeat mechanism and send the ping command to each instance of the cluster every 1 second:

• Subjective offline: If a sentinel node finds that an instance does not respond within the specified time, the instance is considered to be subjectively offline .
• Objective offline: If more than the specified number (quorum) of sentinels consider the instance to be subjectively offline, the instance will be objectively offline . The quorum value should preferably exceed half of the number of Sentinel instances.
  

3.1.3. Cluster failure recovery principle

Once mastera fault is discovered, sentinel needs to select one from the salve as the new master. The selection basis is as follows:

• First, it will determine the length of time the slave node is disconnected from the master node. If it exceeds the specified value ( down-after-milliseconds * 10configured in the configuration file, not equipped with a default value), the slave node will be excluded.
• Then determine slave-prioritythe value of the slave node (configured in the configuration). The smaller the value, the higher the priority. If it is 0, it will never participate in the election.
• If the slave-prority is the same, determine offsetthe value of the slave node. The larger the value, the newer the data and the higher the priority.
• The last step is to determine the size of the slave node 运行id. The smaller the size, the higher the priority.

masterHow to switch after selecting a new one ?
The process is as follows:

• sentinelSend commands to alternative slave1nodes slaveof no one,Let this node bemaster
• sentinelslaveSend slaveof 192.168.150.101 7002 commands to all others to make them slavenew masterslave nodes,Start synchronizing data from the new master。
• at last,sentinelMark the failed node asslave, when the failed node recovers, it will automatically become the slave node of the new master.

3.1.4. Summary

What are the three functions of Sentinel?

• monitor
• failover
• notify

How does Sentinel determine whether a redis instance is healthy?

• Send a ping command every 1 second. If there is no response after a certain period of time, it is considered to be subjectively offline.
• If most sentinels believe that the instance is subjectively offline, the service will be determined to be offline.

What are the failover steps?

• First select a slave as the new master and execute slaveof no one
• Then let all nodes execute slaveof new master
• Modify the faulty node configuration and add slaveof new master

3.2. Build a sentinel cluster【important】

For the specific construction process, please refer to the pre-course material "Redis Cluster.md":

URL: https://editor.csdn.net/md?articleId=128410730

3.3. RedisTemplate (template for actual use of Redis) [important】

RedisTemplateIt is a high degree of encapsulation SpringDataRedisof the pair JedisApi, providing redisvarious operations, exception handling and serialization, and supporting publish and subscribe.

  In the master-slave cluster Sentinelunder cluster supervision Redis, its nodes will change due to automatic failover.RedisThe client must sense this change and update the connection information in a timely manner. SpringThe RedisTemplateunderlying utilization lettuceofNode awareness and automatic switching。

Next, we implement the integrated sentinel mechanism through a test RedisTemplate.
NOTE: To complete this test, first complete 2.1.搭建主从架构the and3.2.搭建哨兵集群

3.3.1. Import Demo project

First, we introduce the projects provided by the pre-class materials Demo:

3.3.2.Introducing dependencies

Dependencies introduced in the project pomfiles :redisstarter

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>

3.3.3.Configure Redis address

  Then application.ymlspecify the relevant information in redisthe configuration file sentinel:
  The Java client can find it based on the address configured below sentinel（哨兵）, thereby knowing Redisthe cluster address.

spring:
  redis:
    sentinel:
      master: mymaster
      nodes:
        - 192.168.150.101:27001
        - 192.168.150.101:27002
        - 192.168.150.101:27003

  For reference (to be consistent), when building a Sentinel cluster, create a sentinel.conffile in the s1 directory. The IP address should be changed to the IP of your own virtual machine. During development, it should be the IP of a different machine.

port 27001  #这里
sentinel announce-ip 192.168.150.101  #这里
sentinel monitor mymaster 192.168.150.101 7001 2  #这里的mymaster
sentinel down-after-milliseconds mymaster 5000
sentinel failover-timeout mymaster 60000
dir "/tmp/s1"

3.3.4. Configure read and write separation

It can be configured in any configuration class:
In the startup class of the project, add a new one bean:

@Bean
public LettuceClientConfigurationBuilderCustomizer clientConfigurationBuilderCustomizer(){
    
    
    return clientConfigurationBuilder -> clientConfigurationBuilder.readFrom(ReadFrom.REPLICA_PREFERRED);
}

or

What is configured in this bean is the read and write strategy, including four types:

• MASTER: Read from the master node
• MASTER_PREFERRED: Read from the master node first, and only read the replica when the master is unavailable.
• REPLICA: read from slave (replica) node
• REPLICA _PREFERRED(Recommended): Read from the slave (replica) node first. Read the master only when all slaves are unavailable.

Start the project:
The browser can read and Rediswrite, and there is also fault detection and recovery because there are sentries;
localhost:8080/get/num
localhost:8080/set/num/666

4. Redis sharded cluster (no sentinel required)

Master-slave and sentry can solve the problems of high concurrent reading and high availability. But there are still two problems unresolved:

• Mass data storage problem
• Problems with high concurrent writing

Master-slave and sentinel solve high-concurrency read operations and existHighly concurrent writingHow to operate;

Using sharded clusters can solve the above problems
Sharded clusters can handle massive data storage and high concurrent write operations
As shown in the picture:

Sharded cluster characteristics:

• There are multiple clusters master, each masterstoring different data.
• Each mastercan have multiple slavenodes
• masterBy pingmonitoring each other's health status (sharded clusters do not require sentinels)
• Client requests can access any node in the cluster and will eventually be forwarded to the correct node.

4.1. Construction of sharded cluster

For the specific construction process, please refer to the pre-course material "Redis Cluster.md":

Article link: https://editor.csdn.net/md?articleId=128410730

4.2. Hash slot【important】

4.2.1.Slot principle

  Redis will map each masternode to a total of 16384 slots ( hash slot) from 0 to 16383. You can see this when viewing the cluster information:

  The data key is not bound to the node, but to the slot [the data follows the slot, and the location of the data can always be found]. redisThe slot value will be calculated based on the valid part of the key, in two cases:

• The key contains "{}", and "{}" contains at least 1 character. The part in "{}" is a valid part.
• The key does not contain "{}", the entire key is a valid part

  For example: if the key is num, then it is calculated based on num. If it is {itcast}num, it is calculated based on itcast. The calculation method is to use the CRC16 algorithm to obtain a hash value, and then take the remainder of 16384, and the result is the slot value.

  As shown in the figure, when executing set a 1 on node 7001, a hash operation is performed on a, and the remainder is taken from 16384. The result is 15495, so it is stored in node 7003.

  After reaching 7003, get numwhen executing, perform a hash operation on num and take the remainder of 16384. The result is 2765, so it is necessary to switch to node 7001.

4.2.2.Testing

Try to connect to node 7001 and store a piece of data:

# 连接
redis-cli -p 7001
# 存储数据
set num 123
# 读取数据
get num
# 再次存储
set a 1

The result is tragic:

when operating the cluster, you need to redis-cliadd -cparameters:

redis-cli -c -p 7001

This time it works:

4.2.3. Summary

How does Redis determine which instance a key should be in?

• Allocate 16384 slots to different instances
• Calculate the hash value based on the valid part of the key, and take the remainder of 16384
• The remainder is used as a slot, just find the instance where the slot is located

How to store the same type of data in the same Redis instance?

• This type of data uses the same valid part, for example, the keys are prefixed with {typeId}

4.3. Cluster scaling【important】

redis-cli --clusterIt provides many commands to operate the cluster, which can be viewed in the following ways:

For example, the command to add a node:

4.3.1.Requirements analysis

Requirement: Add a new masternode to the cluster and store it num = 10

• Start a new redis instance with port 7004
• Add 7004 to the previous cluster and serve as a master node
• Assign a slot to the 7004 node so that numthis keycan be stored in the 7004 instance

Two new features are needed here:

• Add a node to the cluster
• Allocate some slots to new slots

4.3.2. Create a new redis instance【Heavy】

Create a folder 7004:

mkdir 7004

Copy the configuration file to 7004:

cp redis.conf /7004

Modify configuration file:

sed -i /s/6379/7004/g 7004/redis.conf

start up

redis-server 7004/redis.conf

View node status:

ps -ef | grep redis

4.3.3. Add new nodes toredis [Heavy】

The syntax for adding a node is as follows:

execute the command:

# 192.168.150.101:7004是新添加的7004的ip和端口；
# 192.168.150.101:7001是集群中已经存在的ip和端口
redis-cli --cluster add-node  192.168.150.101:7004 192.168.150.101:7001

Check the cluster status through the command:

redis-cli -p 7001 cluster nodes

As shown in the figure, 7004 has joined the cluster and is a master node by default:

However, you can see that the number of slots on node 7004 is 0, so no data can be stored on 7004

4.3.4. Transfer slot【Heavy】

We want to store num to node 7004, so we need to first see what the slot of num is:

As shown in the figure above, numthe slot of 2765.

We can 0~3000transfer the slot from 7001 to 7004. The command format is as follows:

The specific commands are as follows:
Establish connection:

Get the following feedback:

Asked how many slots to move, our plan is 3000: a
new question comes:

who nodewill receive these slots? ?

Obviously yes , so what is the number 7004of 7004 nodes ? Copy this , and then copy it to the console just now: Ask here, where did your slot move from?id

id

• all: represents all, that is, each of the three nodes transfers a part
• Specific id: the id of the target node
• done：no more

Here we want to get it from 7001, so fill in the id of 7001:

After filling in, click done, so the slot transfer is ready:

Are you sure you want to transfer? Enter yes:
Then, view the results through the command:

you can see:

the goal is achieved.

4.4. Failover【important】

The initial status of the cluster is as follows:

7001, 7002, and 7003 are all masters, and we plan to shut down 7002.

4.4.1. Automatic failover

What happens when a master in the cluster goes down?
Stop a redis instance directly, such as 7002:

redis-cli -p 7002 shutdown

1) First, the instance loses connection with other instances

2) Then there is a suspected downtime:

3) Finally, it is confirmed to be offline, and one is automatically promoted slaveto a new one master:

4) When 7002 starts up again, it will become a slave node:

4.4.2. Manual failover

  You can use cluster failovercommands to manually shut down a certain node in the cluster masterand switch to cluster failoverthe slave node that executes the command to achieve imperceptible data migration.. The process is as follows:

This failovercommand can specify three modes:

• default:Default process, as shown in Figure 1~6 steps [Recommended]
• force: Omits the consistency check of offset
• takeover: execute step 5 directly, ignoring data consistency, master status and other masters’ opinions

Case requirements : Perform manual failover on the slave node 7002 to regain the master status

Proceed as follows:

• 1) Use redis-clithe node connection 7002
• 2) Execute cluster failoverthe command

As shown in the picture:

Effect:

4.5. RedisTemplate accesses sharded cluster【important】

RedisTemplateIt is a high degree of encapsulation SpringDataRedisof the pair JedisApi, providing redisvarious operations, exception handling and serialization, and supporting publish and subscribe.

RedisTemplateThe bottom layer is also based on lettucethe implementation of sharded cluster support, and the steps used are basically the same as the sentinel mode:

4.5.1.Introduced redisdependencies starter_

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>

4.5.2. Configure the sharded cluster address

Compared with sentinel mode, only the configuration method of sharded cluster is slightly different, as follows:

spring:
  redis:
    cluster:
      nodes:
        - 192.168.150.101:7001
        - 192.168.150.101:7002
        - 192.168.150.101:7003
        - 192.168.150.101:8001
        - 192.168.150.101:8002
        - 192.168.150.101:8003

4.5.1. Configure read and write separation:

It can be configured in any configuration class:
In the startup class of the project, add a new one bean:

@Bean
public LettuceClientConfigurationBuilderCustomizer clientConfigurationBuilderCustomizer(){
    
    
    return clientConfigurationBuilder -> clientConfigurationBuilder.readFrom(ReadFrom.REPLICA_PREFERRED);
}

Or

the read and write strategies configured in this bean include four types:

• MASTER: Read from the master node
• MASTER_PREFERRED: Read from the master node first, and only read the replica when the master is unavailable.
• REPLICA: read from slave (replica) node
• REPLICA _PREFERRED(Recommended): Read from the slave (replica) node first. Read the master only when all slaves are unavailable.

Start the project:
The browser can read and Rediswrite, and there is also fault detection and recovery because there are sentries;
localhost:8080/get/num
localhost:8080/set/num/666