Detailed Explanation of Redis Sentinel Mechanism (Redis Sentinel) from Entry to Master [Advanced]

Abstract: Introduce in detail the high availability aspects of Redis Advanced, especially the Sentinel mechanism (Redis Sentinel). We will discuss the formation process of the sentinel mechanism, the judgment method of monitoring the main library of the Redis library offline, the selection method of the new main library, and the fault transfer process. At the same time, some reference articles are also provided for readers to further understand.

This article refers to Teacher Jiang Dejun's "Redis Core Technology and Practical Combat-08 | Sentinel Cluster: Can the Master-Slave Database Switch If the Sentinel Is Down?"

In this chapter, we take these questions to learn about Redis high availability

1. What is Redis sentinel mode?
2. What are the advantages of Sentry Mode?
3. How does Sentry Mode work?
4. How to configure the sentinel mode of Redis?
5. How is failover implemented in sentry mode?

0. Preface

Let's first understand that the sentinel mechanism implements the following main functions, and further understand the realization principles of these functions.

1. Monitoring: Sentinel nodes periodically check the health status of Redis master and slave nodes. They send commands and wait for responses to ensure that the Redis instance is functioning properly. If a node does not respond, the sentinel node will mark it as offline.

2. Failure detection and transfer: When a sentinel node detects that the master node is offline, it will broadcast this information to other sentinel nodes. When most sentinel nodes confirm that the master node is offline, they will start to elect a new master node. This process is called failover, and it ensures that the Redis cluster can still function normally if the master node fails.

3. Automatic failover: Once a new master node is elected, the sentinel node will update the configuration of the Redis slave nodes so that they will switch to the new master node. This process is carried out automatically without human intervention.

4. Configuration provisioning and updating: Sentinel nodes monitor the configuration of the Redis cluster and provide it to clients when needed. When the master node fails over, the sentinel nodes update the configuration of the slave nodes so that they are properly connected to the new master node.

5. Provide cluster information: Sentinel nodes can provide information about Redis clusters, such as addresses and statuses of master nodes, slave nodes, and sentinel nodes.

Through these functions, the sentinel mechanism can provide high availability and failover capabilities, ensuring that the Redis cluster is still available when the master node fails.

1. Detailed explanation of the principle

1.1. Formation of sentinel mechanism

Sentinel instances can discover each other thanks to the pub/sub mechanism provided by Redis, that is, the publish/subscribe mechanism. As long as the sentinel has established a connection with the main library, it can publish messages on the main library, such as publishing its own connection information (IP and port). At the same time, it can also subscribe to messages from the main library to obtain connection information published by other sentinels. When multiple Sentinel instances have performed publish and subscribe operations on the main library, they can know each other's IP addresses and ports. In addition to sentinel instances, applications written by ourselves can also publish and subscribe messages through Redis. Therefore, in order to distinguish messages from different applications, Redis will manage these messages by category in the form of channels. The so-called channels are actually categories of messages. When messages are of the same class, they belong to the same channel. Otherwise, it belongs to a different channel. Only applications that subscribe to the same channel can exchange information through published messages.

Take an example of Teacher Jiang Dejun in "Redis Core Technology and Practical Combat"
In the master-slave cluster, there is a __sentinel__:hellochannel named on the master library, through which different sentinels discover each other and communicate with each other. Let me give an example to clarify. In the figure below, Sentinel 1 publishes its own IP (172.16.19.3) and port (26579) to the __sentinel__:hellochannel, and Sentinel 2 and 3 subscribe to the channel. Then at this time, Sentinel 2 and 3 can directly obtain the IP address and port number of Sentinel 1 from this channel. Then, Sentinel 2, 3 can establish a network connection with Sentinel 1. In this way, Sentinel 2 and 3 can also establish a network connection, so that a Sentinel cluster is formed. They can communicate with each other through network connections, such as judging and negotiating whether the main library is offline or not.

Through the above explanation, we can conclude the following points

1. Steps to build a sentinel cluster

• Step 1: Start the sentinel node.
• Step 2: Configure the monitoring object of the Sentinel node.
• Step 3: Communication and coordination among Sentinel nodes.

2. The role and role of sentinel nodes

• Master node: The master node responsible for processing read and write requests.
• Slave node: Data backup and reading are realized by replicating the data of the Master node.
• Sentinel node: monitors the status of the Redis cluster, and is responsible for the offline determination and failover of the main database.

1.1. How does the sentinel know the information of the slave library

The sentinel sends the INFO command to the master library, the master library sends the slave library list to the slave library, and the sentinel establishes a connection with the slave library in the slave library list to monitor the slave library.

1.2. Judgment of main database offline

In Redis Sentinel, the execution of master-slave switching is completed by the cooperation of multiple sentinel nodes in the sentinel cluster. When a sentinel node in the sentinel cluster detects that the master node is offline, it will send a message to other sentinel nodes, and then all sentinel nodes will reach a consensus to elect a new master node and point other slave nodes to the new master node. master node. During the election process, the sentinel node will determine the new master node through voting and heartbeat mechanism. The elected new master node will be broadcast to all clients so that they can communicate with the new master node. Therefore, the execution of master-slave switching is jointly participated by all sentinel nodes.

1. The sentinel node monitors the status of the main library through the heartbeat mechanism.
2. When the sentinel node determines that the main library is offline, it will broadcast to notify other sentinel nodes.
3. The sentinel node determines whether the main library is really offline through the consensus algorithm.

1.3. Sentinel Cluster Election

1. When a sentinel node detects that the master node is offline, it sends a message to other sentinel nodes informing them that the master node has gone offline.
2. After other sentinel nodes receive the message, they also check whether the master node is offline. If they agree that the masternode has gone offline, they enter the election process.
3. During the election process, sentinel nodes will communicate with each other to determine the new master node through voting and heartbeat mechanisms.
4. Each sentinel node will elect a candidate node with the highest priority for itself, and then they will send the candidate node they elect to other sentinel nodes.
5. The sentinel node will select a new master node according to the priority of the candidate node. If there are multiple candidate nodes with the same priority, then it will be decided according to the failover timeout property in the configuration file.
6. The elected new master node will be broadcast to all clients so that they can communicate with the new master node.
   It should be noted that during the election process, the sentinel node needs to reach a majority consensus to elect a new master node. For example, if the total number of Sentinel nodes is 5, then at least 3 Sentinel nodes must agree to the election result to take effect.
   This election process ensures that when the master node goes offline, the sentinel cluster can automatically elect a new master node, thus ensuring the high availability of Redis.

in conclusion

1. When the main library goes offline, the sentinel node will start to elect a new main library.
2. Sentinel nodes select a new main library through a consensus algorithm.
3. After the new main library is selected, the sentinel node will notify other nodes to switch.

The approximate process is shown in the figure

1.4. Failover

1. When the new master library is selected, the sentinel node will switch the slave library to the new master library.
2. The client needs to reconnect to the new master database for data read and write operations.

2. Summary

Through the introduction of this article, we have a detailed understanding of the high availability aspects of Redis Advanced, especially the Sentinel mechanism (Redis Sentinel). We have learned about the formation process of the sentinel mechanism, the judgment method of monitoring the offline of the main library of the Redis library, the selection method of the election of the new main library, and the transfer process of the failure. It is helpful for our study and interview.

Reference article:

1. Redis Sentinel Documentation: https://redis.io/topics/sentinel
2. Teacher Jiang Dejun's "Redis Core Technology and Practice-08 | Sentinel Cluster: Can the master-slave library switch if the sentinel is down?"

3. Redis from entry to proficiency series of articles

"Redis from entry to proficiency [advanced chapter] detailed explanation of redis master-slave replication" "
Redis from entry to proficiency [advanced chapter] Redis transaction detailed explanation"
"Redis from entry to proficiency [advanced chapter] detailed explanation of object mechanism"
"Redis from entry to proficiency [Advanced]: Detailed Explanation of Messaging Publishing and Subscription Mode" "
Redis from Entry to Proficiency [Advanced]: Detailed Explanation of Persistence AOF"
"Redis from Entry to Proficiency [Advanced] Persistence Detailed Explanation of RDB"
"Detailed Explanation of the Underlying Data Structure Dictionary (Dictionary) of Redis from Entry to Proficiency [Advanced Chapter]" "Detailed
Explanation of QuickList of the Underlying Data Structure of Redis from Entry to Proficiency [Advanced Chapter]" "
Redis From Entry to Proficiency [Advanced Chapter] Detailed Explanation of the Simple Dynamic String (SDS) of the Underlying Data Structure" "
Redis from Beginner to Master [Advanced Chapter] Detailed Explanation of the Underlying Data Structure Compression List (ZipList)" "
Redis From Beginner to Master [Advanced Chapter] 】Detailed explanation and usage examples of the data type Stream

4. Redis sentinel mode interview questions

Based on the above learning, let's briefly answer common Redis high availability related interview questions

4. 1. What is the sentinel mode of Redis?

Redis' sentinel mode is a mechanism for achieving high availability. In sentinel mode, multiple Redis instances form a sentinel cluster, one of which acts as the master server and the other instances act as slave servers. Sentinel monitors the status of the master server and automatically promotes a slave server to the new master server when the master server is down to ensure continuous availability of the system.

4. 2. What are the advantages of Sentry mode?

• Automatic failover: When the primary server goes down, Sentinel can automatically elect a new primary server to ensure high availability of the system.
• Real-time monitoring: Sentinel can monitor the status of Redis instances in real time, and take timely measures when abnormalities are found, such as automatically switching the main server.
• Flexibility: Sentinel mode can configure the size of the Sentinel cluster according to actual needs to adapt to systems of different scales.

4. 3. How does Sentry Mode work?

• Sentinels in a Sentinel cluster constantly monitor the state of the Redis instance.
• Each sentinel detects the liveness of the Redis instance by sending a PING command.
• If a sentinel finds that the master server is down, it will elect a new master server through the election algorithm and broadcast this information to other sentinels.
• After receiving the broadcast, other Sentinels will also update their configurations to adapt to the new master server.
• The client can obtain the current main server address by connecting to any sentinel, and establish a connection with the main server.

4. 4. How to configure the sentinel mode of Redis?

• Set parameters in the configuration file of each Redis instance sentinelto specify the relevant configuration of the sentinel mode.
• Start each Redis instance and specify the corresponding configuration file.
• Run the command on any Sentinel in the Sentinel cluster redis-sentinelto start the Sentinel process.
• The Sentinel process automatically discovers other Sentinel and Redis instances and starts monitoring their status.

4. 5. How is failover implemented in sentinel mode?

• When a Sentinel discovers that the master server is down, it sends election requests to other Sentinels.
• All sentinels in the sentinel cluster will elect a new master server.
• Election rules are usually evaluated based on factors such as priority, last replicated offset, etc.
• After the election is complete, the new master server address will be broadcast to other sentries and clients.
• The client can obtain the address of the new master server by connecting to any Sentinel, and establish a connection with it.

Hello everyone, I'm Freezing Point, and today's high-availability sentinel mechanism (Redis Sentinel) is explained in detail, and that's all. If you have questions or opinions, you can leave a message in the comment area.