redis--------sentinel mode

1. Sentry Mode

        

 

 Just imagine, if in the master-slave mode, the master node hangs up in the middle of the night, the operation and maintenance gets up from the bed in a daze, turns on the computer, and manually upgrades, I am afraid that it will be in the headlines the next day.

        The emergence of the sentinel mode is used to solve the problem that the master node cannot be automatically upgraded in the master-slave mode. A sentinel is a node used to monitor the health of the master-slave node. When the master node hangs up, it automatically selects an optimal slave node to upgrade to master node.

        But what if the sentinel hangs up? Therefore, Sentry is generally a cluster, which is the heart of high availability of the cluster, and generally consists of 3-5 nodes. Even if individual nodes fail, the cluster can still run normally.

advantage

• Can automatically elect a new master after the master hangs up

shortcoming

• The master hangs up, and switching to a new master will cause the loss of data that cannot be synchronized between the master and the slave in the future
• Big data is highly concurrency, and there is still an upper limit on the memory of a single master
• Master-slave full synchronization still takes a lot of time
• A single Redis can only use a single core of the CPU, and it is difficult to deal with massive data

Solution

• Redis cluster solution

  When the client connects to Redis, it will first connect to Sentinel, query the address of the master through Sentinel, and then connect to the master for data interaction. When the master hangs up, the client asks Sentinel for the master address again and connects to the new master.

        As can be seen from the figure above, the master hangs up, the original master-slave replication is disconnected, and the client and master are also disconnected. Then a slave becomes the new master, and performs new master-slave replication with the rest of the slaves. The client continues to interact through the new master. Sentinel continues to monitor the old master that has been suspended. Once the old master recovers, the cluster will become In the figure below, the old master becomes the new slave, and a master-slave replication relationship is established from the new master.

2. Working Mechanism of Sentry Mode

• A Sentinel or Sentinel cluster can manage multiple master-slave Redis, if there is only one Redis, Sentinel is meaningless
• The master hangs up, Sentinel elects one of the slaves to upgrade to the master, and modifies the configuration file
• After the suspended master is recovered, it can only be a slave and replicate with the newly elected master master-slave
• Sentinel may hang, so Sentinel is generally a cluster
• Each Sentinel node regularly detects whether the Redis data node and the rest of the Sentinel nodes are reachable
• The failure of the node is completed by multiple Sentinel nodes, effectively preventing misjudgment
• Redis master-slave replication is asynchronous. If the master hangs up, the slave node may lose some information. Sentinel cannot guarantee that the message will not be lost at all, but it can do as little as possible.
• The Sentinel.slave_for method can take out a slave address from the connection pool using the polling scheme to achieve load balancing

3.1 Scheduled monitoring tasks

• Every 10 seconds, each Sentinel node will send an info command to the master node and the slave node to obtain the latest topology, which can sense the newly added or failed Redis data node
• Every 2 seconds, each Sentinel node will communicate with other Sentinel nodes, can discover new Sentinel nodes, and exchange judgment information on the master node with other Sentinel nodes to facilitate failover and election
• Every 1 second, each Sentinel node will send a ping to the master node, slave node and other Sentinel nodes to do a heartbeat detection to confirm whether these nodes are reachable, and to monitor each node
   

3.2 Subjective offline and objective offline 

subjective offline

When the Sentinel node pings other nodes and fails to reply after a timeout, the Sentinel node will make a failure judgment for the node. Subjective offline is the opinion of the current Sentinel node, and there is a possibility of misjudgment 

Go offline objectively 

When the Sentinel subjectively offline node is the master, the Sentinel will ask other Sentinels to judge the master. When most Sentinels agree with the master’s offline judgment, then this judgment is more objective 

 3.3 Leader Election of Sentinel Cluster

The Sentinel node has objectively offlined the master, but failover cannot be performed immediately, because only one Sentinel node is required to complete the failover work, so a leader must be selected in the Sentinel cluster for failover.

Redis uses the Raft algorithm to implement leader election:

• Every online Sentinel node is eligible to be a leader, he asks other nodes to vote for himself
• Vote for whoever requests it first, but not yourself
• The first to get a majority (i.e. half + 1) of the votes to be elected as the leader

 3.4 Failover

1. Exclude slave nodes that are subjectively offline
2. Select a slave node with high priority, if not, proceed to 3
3. Select the slave node with the largest copy offset (the most complete copy), if there is no more 4
4. Select the slave node with the smallest runid

4. The principle of sentinel mode

1 多个sentinel发现并确认master有问题
2 选举触一个sentinel作为领导（raft算法）
3 选取一个slave作为新的master
4 通知其余slave成为新的master的slave
5 通知客户端主从变化
6 等待老的master复活成为新master的slave

 Sentinel: It is a process --- "Start using redis-sentinel to start, with its own configuration file. After starting, you can connect to the client (redis-cli) to view the status of the sentinel

4.1 Configure sentry (one master and two slave architecture --- "running multiple sentries --" generally an odd number (3))

 Configure the sentinel configuration file:

port 26379
daemonize yes
dir /root/lqz/redis-6.2.9/data
protected-mode no
bind 0.0.0.0
logfile "redis_sentinel.log"

sentinel monitor mymaster 127.0.0.1 6379 2
sentinel down-after-milliseconds mymaster 30000
sentinel parallel-syncs mymaster 1
sentinel failover-timeout mymaster 180000
# 2 配置3个配置文件，监听端口不一样(正常应该放在3台机器上)

# 3 启动哨兵
#启动三个哨兵
./src/redis-sentinel sentinel_26379.conf
./src/redis-sentinel sentinel_26380.conf
./src/redis-sentinel sentinel_26381.conf


# 4 演示故障
	-把主库停止
  -查看自动故障切换
  -启动起原来主库，变成了从

4.2python operation sentinel

import redis
from redis.sentinel import Sentinel

# 连接哨兵服务器(主机名也可以用域名)
# 10.0.0.101:26379
sentinel = Sentinel([('8.130.125.9', 26379),
                     ('8.130.125.9', 26380),
                     ('8.130.125.9', 26381)
         ],
                    socket_timeout=5)

print(sentinel)
# 获取主服务器地址
master = sentinel.discover_master('mymaster')
print(master)

# 获取从服务器地址
slave = sentinel.discover_slaves('mymaster')
print(slave)



##### 读写分离
# 获取主服务器进行写入
master = sentinel.master_for('mymaster', socket_timeout=0.5)
w_ret = master.set('foo', 'bar')

# slave = sentinel.slave_for('mymaster', socket_timeout=0.5)
# r_ret = slave.get('foo')
# print(r_ret)