Redis master-slave replication, sentinel, cluster

Master-slave replication:

• Master-slave replication is the basis of high-availability Redis, and sentinels and clusters are highly available on the basis of master-slave retesting.
• Master-slave replication mainly implements multi-level backup of data, as well as load balancing and simple failure recovery for read operations.
• A master host can have multiple slave slaves. And one slave can have multiple slaves. In this way, a powerful multi-level server cluster architecture (high scalability) is formed. It can avoid Redis single point of failure and achieve disaster recovery effect (high availability). Read-write-separated architecture to meet concurrent application scenarios with more reads and less writes
• Defects: Failure recovery cannot be automated, write operations cannot be load balanced, and storage capacity is limited by a single machine.

sentinel:

• On the basis of master-slave replication, Sentinel realizes automatic failure recovery.
• Defects: The write operation cannot be load balanced, the storage capacity is limited by a single machine, and the sentinel cannot fail over the slave node. In the read-write separation scenario, the slave node failure will cause the read service to be unavailable, and additional monitoring of the slave node is required , Switch operation.

Cluster:

• Through clustering, Redis solves the problem that write operations cannot be load balanced, and storage capacity is limited by a single machine, and realizes a relatively complete high-availability solution.

1. Redis master-slave replication

• Master-slave replication refers to copying the data of one Redis server to other Redis servers. The former is called the master node (Master), and the latter is called the slave node (Slave). The data replication is one-way and can only be from the master node to the slave node.
• By default, each Redis server is the master node, and a master node can have multiple slave nodes (or no slave nodes), but a slave node can only have one master node.

1. The role of master-slave replication

effect	Explanation
Data redundancy	Master-slave replication realizes the hot backup of data, which is a data redundancy method besides persistence.
Recovery	When the master node has a problem, the slave node can provide services to achieve rapid failure recovery: in fact, it is a kind of service redundancy.
Load balancing	On the basis of master-slave replication, with the separation of read and write, the master node can provide the write service, and the slave node can provide the read service (that is, the application connects to the master node when writing Redis data, and the application connects to the slave node when reading Redis data), sharing the server Load; especially in the scenario of writing less and reading more, sharing the read load by multiple slave nodes can greatly increase the concurrency of the Redis server.
High-availability cornerstone	In addition to the above functions, master-slave replication is also the basis for the implementation of sentinels and clusters, so master-slave replication is the basis for Redis's high availability.

2. Master-slave replication characteristics

• 1. The master can have multiple slaves
• 2. Multiple slaves can be connected to the same master, and can also be connected to other slaves
• 3. Master-slave replication will not block the master. When synchronizing data, the master can continue to process client requests
• 4. Improve the scalability of the system
• 5. You can disable data persistence in the master, comment out all save configurations in the master configuration file, and just configure data persistence on the slave

3. Master-slave replication process

• 1. If a slave machine process is started, it will send a "sync command" command to the Master machine to request a synchronous connection.
• 2. Whether it is the first connection or reconnection, the Master machine will start a background process to save the data snapshot to the data file (execute the rdb operation), and Masterf will record all the commands to modify the data and cache it in the data file.
• 3. After the background process completes the cache operation, the Master machine will send the data file to the Slave machine, and the Slave machine will save the data file to the hard disk, and then load it into the internal f, and then the Master machine will modify all the data The operation is sent to the slave machine. If the Slave fails and causes a downtime, it will automatically reconnect after returning to normal.
• 4. After the Master machine receives the connection from the Slave side machine, it sends its complete data file to the Slave side machine. If Mater receives synchronization requests from multiple slaves at the same time, the laster will start a process in the background to save the data The file is then sent to all slave-side machines to ensure that all slave-side machines are normal.

4. Redis master-slave replication construction experiment

1. Experiment preparation:

Host	IP address	software
Master node	192.168.23.128	Redis
Slave1 node	192.168.23.129	Redis
Slave2 node	192.168.23.130	Redis

2. Modify the Redis configuration file (Master node)

vim /etc/redis/6379.conf
bind 0.0.0.0                      #70行修改监听地址为0.0.0.0，监听所有
daemonize yes                     #137行，开启守护进程
logfile /var/log/redis_6379.log   #172行，指定日志文件目录
dir /var/lib/redis/6379           #264行，指定工作目录
appendonly yes                    #700行，开启AOF持久化功能

/etc/init.d/redis_6379 restart    #重启服务

3. Modify the Redis configuration file (Slave node)

vim /etc/redis/6379.conf
bind 0.0.0.0                      #70行，修改监听地址为0.0.0.0，监听所有
daemonize yes                     #137行，开启守护进程
logfile /var/log/redis_6379.log   #172行，指定日志文件目录
dir /var/lib/redis/6379           #264行，指定工作目录
replicaof 192.168.23.128 6379     #288行，指定要同步的Master节点IP和端口
appendonly yes                    #701行，开启AOF持久化功能

/etc/init.d/redis_6379 restart    #重启服务

4. Verify the master-slave effect

• Look at the log on the Master node
  tail -f /var/log/redis_6379.log
  

• Verify the slave node on the master node

redis-cli info replication

#Replication
role:master
connected_slaves:2
slave0:ip=192.168.23.129,port=6379,state=online,offset=1986,lag=1
slave1:ip=192.168.23.130,port=6379,state=online,offset=1986,lag=1

• Create a key verification on the master master
  
• Query on Slave, you can only query, not create

Two. Redis sentinel mode

1. The core function of the sentinel

On the basis of master-slave replication, Sentinel introduces automatic failover of the master node.

2. The principle of sentinel mode

It is a distributed system used to monitor each server in the master-slave structure. When a failure occurs, a new Master is selected through a voting mechanism, and all slaves are connected to the new Master. Therefore, the number of clusters running sentinels should not be less than three nodes.

3. The role of sentinel mode

effect	Explanation
monitor	The sentinel will constantly check whether the master node and slave node are operating normally
Automatic failover	When the master node fails to work normally, the sentinel will start an automatic failover operation. It will upgrade one of the slave nodes of the failed master node to a new master node, and make other slave nodes copy the new master node
Notification (reminder)	The sentinel can send the result of the failover to the client

4. The constituent nodes of the sentinel

node	Explanation
Sentinel node	The sentinel system consists of one or more sentinel nodes, which are special redis nodes and do not store data
Data node	Both the master node and the slave node are data nodes

5. The activation of the sentry

• The startup of the sentinel depends on the master-slave mode, so you must install the master-slave mode before going to the sentinel mode. All nodes need to deploy the sentinel mode. The sentinel mode will monitor whether all Redis working nodes are normal. When the Master appears When there is a problem, because other nodes have lost contact with the master node, they will vote. More than half of the vote is considered that there is a problem with this Master, and then the sentry room will be notified, and then one of the Slaves will be selected as the new Master.
• It is important to note that objective offline is a concept unique to the master node; if the slave node and the sentinel node fail, after the sentinel subjectively offline, there will be no subsequent objective offline and failover operations.

6. Redis sentinel mode build experiment

1. Experiment preparation:

Host	IP address	software
Master node	192.168.23.128	Redis
Slave1 node	192.168.23.129	Redis
Slave2 node	192.168.23.130	Redis

2. Modify the configuration file of Redis sentry mode (all node operations)

vim / opt/redis-5.0.7/ sentinel.conf
protected-mode no                                     #17行,关闭保护模式
port 26379                                            #21行，Redis哨兵默认的监听端口
daemonize yes                                         #26行，指定sentinel为后合启动
logfile "/var/ log/ sentinel. log"                    #36行,指定日志存放路径
dir "/var/ lib/redis/6379"                            #65行，指定数据库存放路径
sentinel monitor mymaster 192.168.23.128 6379 2       #84行，修改指定该哨兵节点监控192.168.23.128 6379这个主节点，该主节点的名称是mymaster，最后的2的含义与主节点的故障判定有关:至少需要2个哨兵节点同意，才能判定主节点故障并进行故障转移
sentinel down-after-milliseconds mymaster 3000        #113行，判定服务器down掉的时间周期，默认3000毫秒（30秒)
sentinel failover-timeout mymaster 180000             #146行，故障节点的最大超时时间为180000 (180秒)

3. Activate sentry mode

先启动Master，再启动Slave
cd /opt/redis-5.0.7/
redis-sentinel sentinel.conf &    #"&"表示后台运行

4. View sentry information

redis-cli -p 26379 info Sentinel

# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
sentinel_simulate_failure_flags:0
master0:name=mymaster,status=ok,address=192.168.23.128:6379,slaves=2,sentinels=3

5. Failure simulation

• View redis-server process number

ps -elf | grep redis

5 S root       8458      1  0  80   0 - 39869 ep_pol 15:46 ?        00:00:29 /usr/local/redis/bin/redis-server 0.0.0.0:6379
5 S root       9933      1  0  80   0 - 38461 ep_pol 17:33 ?        00:00:18 redis-sentinel *:26379 [sentinel]
0 S root      10519   9726  0  80   0 - 28169 pipe_w 18:21 pts/3    00:00:00 grep --color=auto redis

• Kill the redis-server process number on the Master node

kill -9 8458             #Mster节点上redis-server进程号
ps -elf | grep redis     #再次查看进程

• Validation results

tail -f /var/log/sentinel.log   #查看哨兵的日志

9933:X 25 Feb 2021 18:29:32.895 * +failover-state-send-slaveof-noone slave 192.168.23.130:6379 192.168.23.130 6379 @ mymaster 192.168.23.129 6379
9933:X 25 Feb 2021 18:29:32.954 * +failover-state-wait-promotion slave 192.168.23.130:6379 192.168.23.130 6379 @ mymaster 192.168.23.129 6379
9933:X 25 Feb 2021 18:29:33.694 # +promoted-slave slave 192.168.23.130:6379 192.168.23.130 6379 @ mymaster 192.168.23.129 6379
9933:X 25 Feb 2021 18:29:33.694 # +failover-state-reconf-slaves master mymaster 192.168.23.129 6379
9933:X 25 Feb 2021 18:29:33.774 # +failover-end master mymaster 192.168.23.129 6379
9933:X 25 Feb 2021 18:29:33.774 # +switch-master mymaster 192.168.23.129 6379 192.168.23.130 6379
9933:X 25 Feb 2021 18:29:33.775 * +slave slave 192.168.23.128:6379 192.168.23.128 6379 @ mymaster 192.168.23.130 6379
9933:X 25 Feb 2021 18:29:33.775 * +slave slave 192.168.23.129:6379 192.168.23.129 6379 @ mymaster 192.168.23.130 6379
9933:X 25 Feb 2021 18:30:03.839 # +sdown slave 192.168.23.129:6379 192.168.23.129 6379 @ mymaster 192.168.23.130 6379
9933:X 25 Feb 2021 18:30:03.840 # +sdown slave 192.168.23.128:6379 192.168.23.128 6379 @ mymaster 192.168.23.130 6379

• View the sentry information again on selve

redis-cli -p 26379 info Sentinel

# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
sentinel_simulate_failure_flags:0
master0:name=mymaster,status=ok,address=192.168.23.130:6379,slaves=2,sentinels=3

Three. Redis cluster mode

• Cluster, or Redis cluster, is a distributed storage solution introduced in Redis 3.0.
• The cluster is composed of multiple nodes (Node), and Redis data is distributed among these nodes. The nodes in the cluster are divided into master nodes and slave nodes: only the master node is responsible for the protection of read and write requests and cluster information; the slave nodes only copy the data and status information of the master node.

1. The role of clusters

effect	Explanation
Data partition	Also known as data sharding is the core function of the cluster. The cluster disperses data to multiple nodes. On the one hand, it breaks through the limitation of Redis single machine memory size and greatly increases the storage capacity; on the other hand, each master node can provide external read and write services, which greatly improves the responsiveness of the cluster.
High availability	The cluster supports master-slave replication and automatic failover of the master node (similar to sentinel); when any node fails, the cluster can still provide external services.

2. Data sharding of the Redis cluster

• Redis cluster introduces the concept of hash slot.
• The Redis cluster has 16384 hash slots (each node of the cluster numbered 0-163833 is responsible for a part of the hash slots.
• After each Key is verified by CRc16, the remainder of 16384 is taken to determine which hash slot to place, and this value is used to find the node corresponding to the corresponding slot, and then directly jump to the corresponding node for access operations. .

3. Take a cluster composed of three nodes as a column

node	Hash slot value
Node A	Contains hash slots 0-5460
Node B	Contains hash slots 5461-10922
Node C	Contains hash slots 10923 to 16383

4. Master-slave replication model of Redis cluster

• The cluster has three nodes A, B, and c. If node B fails, the entire cluster will be unusable due to lack of slots in the range of 5461-10922.
• Add a slave node A1, B1, and c1 for each node. The entire cluster consists of three Nester nodes and three slave nodes. After node B fails, the cluster elects the master node with one bit to continue to serve. When B and B1 both fail, the cluster will be unavailable.

5. Redis cluster mode construction experiment

• A redis cluster generally requires 6 nodes, 3 masters and 3 slaves. For convenience, all nodes here are simulated on the same server: distinguished by port number: 3 master node port numbers: 6001/6002/6003, corresponding slave node port numbers: 6004/6005/6006.

1. Create

cd /etc/redis/
mkdir -p redis-cluster/redis600{1..6}   #递归创建工作目录
ls -r      #查看递归目录是否创建成功
cd /opt/redis-5.0.7/src/

for i in {1..6}
do
cp /opt/redis-5.0.7/redis.conf /etc/redis/redis-cluster/redis600$i
cp /opt/redis-5.0.7/src/redis-cli /opt/redis-5.0.7/src/redis-server /etc/redis/redis-cluster/redis600$i
done

2. Turn on the cluster function, and modify the configuration files of the other 5 folders by analogy. Note that the 6 ports are all different.

cd /etc/ redis/redis-cluster/redis6001

vim redis.conf
#bind 127.0.0. 1                       #69行，注释掉bind项，默认监听所有网卡
protected-mode no                      #88行,修改,关闭保护模式
port 6001                              #92行,修改，redis监听端口，
daemonize yes                          #136行,开启守护进程,以独立进程启动
appendonly yes                         #699行,修改,开启AoF持久化
cluster-enabled yes                    #832行,取消注释,开启群集功能
cluster-config-file nodes-6001.conf    #840行,取消注释,群集名称文件设置
cluster-node-timeout 15000             #846行,取消注释群集超时时间设置

#因为还有5个配置文件需要改很麻烦，可以把更改好的这个复制给其余5个这样就只需要改92行监听端口和840行文件名称就可以了不需要全部重新改一次了
for i in {2..6}
> do
> /usr/bin/cp -f redis.conf ../redis600$i/redis.conf
> done

3. Start the Redis node

分别进入之前创建的6个文件夹，启动Redis节点
cd /etc/redis/redis-cluster/redis6001
redis-server redis.conf
for i in {1..6}
do
cd /etc/redis/redis-cluster/redis600$i
redis-server redis.conf
done
ps -elf | grep redis             #查看Redis进程

4. Start the cluster

redis-cli --cluster create 127.0.0.1:6001 127.0.0.1:6002 127.0.0.1:6003 127.0.0.1:6004 127.0.0.1:6005 127.0.0.1:6006 --cluster-replicas 1
#六个实例分为三组，每组一主一从，前面的做主节点，后面的做从节点。下面交互的时候需要输入yes才可以创建

5. Test the cluster

redis-cli -p 6001 -c            #-c参数表示节点之间可以互相跳转
127.0.0.1:6001> cluster slots   #查看节点的哈希槽范围
1. 1) (integer) 10923
   2) (integer) 16383           #哈希槽编号范围
   3) 1) "127.0.0.1"            #主节点ip
      2) (integer) 6003         #主节点端口
      3) "ac76c0f001e19c826343ac41ce9058541da37fca"
   4) 1) "127.0.0.1"            #主节点IP
      2) (integer) 6004         #主节点端口
      3) "b7c422177980998c21ec416287b6ec8873bdfb97"
2. 1) (integer) 5461
   2) (integer) 10922
   3) 1) "127.0.0.1"
      2) (integer) 6002
      3) "7687a2c7e2b40071981b2a6b2d6a29c0d8207c16"
   4) 1) "127.0.0.1"
      2) (integer) 6006
      3) "d7b58172c18df14c8338c9a2ae6671596972ca36"
3. 1) (integer) 0
   2) (integer) 5460
   3) 1) "127.0.0.1"
      2) (integer) 6001
      3) "9892237a65dccdb81d64c01d2003cef5fc694fc6"
   4) 1) "127.0.0.1"
      2) (integer) 6005
      3) "b20f379155c32df5b1698e888aa3b403178d50dd"

127.0.0.1:6001> set ll 123              #在6001创建一个键
-> Redirected to slot [9069] located at 127.0.0.1:6002
OK
127.0.0.1:6002> CLUSTER KEYSLOT ll      #查看ll键的槽位号
(integer) 9069

redis-cli -p 6006 -c
127.0.0.1:6006> KEYS *                  #对应的从服务器节点也有这个键，别的节点没有
1) "ll"