Redis master-slave principle + sentry mode

1. Principle of master-slave replication

①: The slave server establishes a long socket connection before the master server master.
②: The slave server slave sends a PSYNC command to the master server master to request data replication.
③: After the master server master receives the PSYNC command, it will use the sub-thread to generate the latest rdb snapshot file through the bgsave command and send it to the slave server slave.
During persistence, the master will continue to receive client requests, and it will cache these requests that may modify the data set in the memory repl buffer
④: The slave clears useless data, and accepts the data from the master to load into the memory
⑤: master Then send the command that was cached in the memory during the previous persistence to the slave.
⑥: The slave accepts the new command sent by the master and executes it
⑦: At this time, the data has been synchronized, and when the master has a new write operation, it will continue to be sent to the slave through the socket long connection to ensure the consistency of the master-slave data!
Note: If the master receives multiple concurrent connection requests from multiple slaves, it will only persist once, not one connection once, and then send this piece of persistent data to multiple concurrently connected slaves.

1. What if the slave node hangs up during the master-slave transmission?

When Salve hangs up when it receives half of the data due to the network and other reasons, after a period of time, the salve slave node is artificially restarted, then how is the data transmission handled at this time?
Answer: Starting from the redis 2.8 version, redis uses the command PSYNC that supports partial data replication to synchronize data with the master. The slave and master can only perform partial data replication (resumed transmission) after the network connection is disconnected and reconnected. The flowchart is as follows:

①: First, redis will open a cache pool by default when it is running, which is used to cache the latest redis commands. You can configure
repl-backlog-size 1mb in 6379.conf ####redis command cache pool, the default size 1Mb
②: When the slave is disconnected from the master and the connection is re-established, the slave will send a PSYNC command to the master, and use the offset offset to locate the position of the data transmission when the connection is disconnected, and start from this position to continue the transfer
③: If the slave node is disconnected for too long and the offset is too old, and the corresponding position cannot be found in the command buffer pool in the master, then a full data copy will be performed. Can't use breakpoint to resume upload!

2. What is a master-slave replication storm?

Master-slave replication storm: Multiple slave nodes replicate the master node at the same time, causing excessive pressure on the master node.
In order to solve the problem of master-slave replication storm, some slave nodes can synchronize data with slave nodes. The architecture is designed as follows:

3. Advantages and disadvantages of master-slave replication

1. Advantages

2. Disadvantages

Two, Redis project deployment

1. Project topology

2. Environment

 一台主服务器  master ：192.168.1.10
两台备服务器   slave1 ：192.168.1.11
             slave2 ：192.168.1.12

1. Install redis

Both the
active and standby servers need to be installed. Transfer the redis installation package via xshell files

1. Unzip

[root@master ~]# tar zxvf redis-5.0.4.tar.gz   
[root@master ~]# cd redis-5.0.4/
[root@master redis-5.0.4]# make 配置安装
[root@master redis-5.0.4]# make DREFIX=/usr/local/redis install
更改安装路径可以用make PREFIX=安装路径 install
[root@master redis-5.0.4]# cd

2. Create a link

[root@master ~]# ln -s /usr/local/redis/bin/* /usr/local/bin/    
[root@master ~]# cd redis-5.0.4/utils/    
[root@master utils]# ls -lh    查看安装脚本
[root@master utils]# ./install_server.sh    运行脚本
[root@master utils]# netstat -anpt | grep redis   查看端口状态

After the prompt installation is successful, the basic configuration of Redis is complete

2. Configure Redis master-slave replication service

On the master server

[root@master utils]# cd
[root@master ~]# vi /etc/redis/6379.conf 
[root@master ~]# /etc/init.d/redis_6379 stop    #服务关闭
[root@master ~]# /etc/init.d/redis_6379 start    #服务开启
修改添加
bind 0.0.0.0                     修改监听地址为0.0.0.0
daemonize yes                    开启守护进程
logfile /var/log/redis_6379.log  修改日志文件目录
dir /var/lib/redis/6379          修改工作目录
appendonly yes                   开启AOF持久化功能

Edit configuration on device 1, 2

[root@slave1 utils]# vi /etc/redis/6379.conf 
[root@slave1 utils]# /etc/init.d/redis_6379 restart  服务重启
修改添加
bind 0.0.0.0                 修改监听地址为0.0.0.0
appendonly yes               开启AOF持久化功能
replicaof 192.168.1.10 6379     增加一个同步master节点IP和端口

View on the master server to
verify the master-slave effect

[root@master ~]# tail -f /var/log/redis_6379.log 

Same as above for standby 2 server

Test the master-slave data replication function
on the master server

[root@master ~]# redis-cli   #进入数据库
127.0.0.1:6379> set fa a     #设置fa键得值为a
127.0.0.1:6379> get fa       #查看fa键得值
127.0.0.1:6379> info replication   #信息复制，状态信息

On the standby 1 server

[root@slave1 ~]# redis-cli    #进入数据库
127.0.0.1:6379> get fa        #获取fa键得值
127.0.0.1:6379> info replication     #信息复制，状态信息

On the standby 2 server

// An highlighted block
var foo = 'bar';

3. After the simulated main server hangs

On the master server, turn off the service

[root@master ~]# /etc/init.d/redis_6379 stop      关闭服务
[root@master ~]# tail -f /var/log/redis_6379.log  查看日志

Check the phenomenon on the standby 1 server

[root@slave1 ~]# tail -f /var/log/redis_6379.log 查看日志
[root@slave1 ~]# redis-cli                       进入数据库
127.0.0.1:6379> get k                            获取k的键值                    
"aa"                                                  
127.0.0.1:6379> info replication               信息复制，状态信息
# Replication
role:slave                                    状态：从服务器
master_host:192.168.1.10        

On the standby 2 server,

you can see that after the main server is shut down, the status of the slave server will not be converted to the master server, but the data will be copied and transferred, and it can be read and viewed.

Solution:
Through the sentinel function, the state of the master and slave can automatically switch when the master server fails

First restore to the previous state on the
master and
resume online

[root@master ~]# /etc/init.d/redis_6379 start      服务启动
[root@master ~]# tail -f /var/log/redis_6379.log   查看日志
[root@master ~]# redis-cli                         进入数据库
127.0.0.1:6379> info replication            信息复制，状态查看
# Replication
role:master          主服务 
connected_slaves:2   连接2个服务器
slave0:ip=192.168.1.11,port=6379,state=online,offset=84,lag=1
slave1:ip=192.168.1.12,port=6379,state=online,offset=84,lag=0
master_replid:087088c1aa398b11c1e4759e93eb8abb6bc277e2
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:84
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:84
127.0.0.1:6379> set nb b      添加nb键值为b
OK
127.0.0.1:6379> get nb        获取nb的键值
"b"

On the standby 1 server

[root@slave1 ~]# tail -f /var/log/redis_6379.log   #查看日志
[root@slave1 ~]# redis-cli             

Three, the principle of sentinel mode

• The sentinel sentinel is a special redis service, which does not provide read and write services. It is mainly used to monitor the status of redis instance nodes!
• Under the sentinel architecture, when the client requests the redis service for the first time, it will first find the redis master node from the sentinel, and then directly access the redis master node. It will not access the redis master node through the sentinel agent every time. When the master node changes, the sentinel will perceive it for the first time, and notify the client of the new redis master node (the redis client side generally implements the subscription function and subscribes to the node change message published by sentinel)
• 

1. The main role of the sentry

1. Phenomenon

When the master node hangs up, the server console will print a connection timeout error. After a period of time, it returns to normal and you can continue to write data to redis!

2. Reason

• The reason is that the sentry will always monitor the master node. When the master node is down, the server console will print a connection timeout error. But at the same time, the sentry confirms that the master is down through half of the mechanism, and will elect a slave as the new master
• During the election period, the console will still print errors. Once the election is successful, the normal connection will be restored. This is also the reason for the above phenomenon! !
• When the originally hung master node is restored, it will automatically be called the new master cluster node, completing the sentinel high-availability architecture!

The sentinel should be set to an odd number, at least three, which involves voting!

2. Sentinel deployment process

Edit the configuration file of sentry mode on the master server

[root@master ~]# vi redis-5.0.4/sentinel.conf 
修改添加
protected-mode no               关闭安全模式
daemonize yes                   指定sentine1为后台启动，开启守护进程
logfile "/var/log/sentinel.log" 指定日志存放路径
dir /var/lib/redis/6379         指定数据库存放路径

sentinel monitor mymaster 192.168.1.10 6379 2     
指定几个哨兵（slave）检测主服务器故障，才会进行故障迁移（主服务器ip地址，端口号，slave数）

sentinel down-after-milliseconds mymaster 3000  
 判定服务器down掉的时间周期，默认30000毫秒（30秒）
 
sentinel failover-timeout mymaster 100000
故障节点的最大超时时间为100000毫秒（100秒）

Copy files to slave1 and slave2

[root@master ~]# scp redis-5.0.4/sentinel.conf root@192.168.1.11:/root/redis-5.0.4
[root@master ~]# scp redis-5.0.4/sentinel.conf root@192.168.1.12:/root/redis-5.0.4

Start sentinel mode
, start the master server first, then start the slave server

[root@master ~]# redis-sentinel redis-5.0.4/sentinel.conf &
[1] 69742
[root@slave1 ~]# redis-sentinel redis-5.0.4/sentinel.conf &
[1] 62685
[root@slave2 ~]# redis-sentinel redis-5.0.4/sentinel.conf &
[1] 62250

1. View the log on the master server

[root@master ~]# tail -f /var/log/sentinel.log 

2. View process status

[root@master ~]# ps aux | grep redis
[root@master ~]# ps aux | grep sentinel

Indicates that the sentry service has been started
3. Remotely log in to the database to view the status of the sentry

[root@master ~]# redis-cli -h 192.168.1.10 -p 26379 info sentinel
[root@master ~]# redis-cli -h 192.168.1.10 -p 6379 info replication

1. Simulate a problem with the primary server and down, check the status change of the secondary server

On the master server

[root@master ~]# /etc/init.d/redis_6379 stop  停止服务
[root@master ~]# ps aux | grep redis        查看进程状态

View logs on the slave server

[root@slave1 ~]#  tail -f /var/log/sentinel.log 

Found that the master state is transferred to the slave 1

View remote login in slave2 to view sentry information

[root@slave2 ~]# redis-cli -h 192.168.1.11 -p 26379 info sentinel

2. View the address of the main server detected by the sentinel previously configured

On the standby 1 server

[root@slave1 ~]# vi redis-5.0.4/sentinel.conf 

The configuration address changes found on the main server will also automatically change the detection address when the main state is automatically switched

3. View the status changes when the original master server comes back online

On the master server

[root@master ~]# /etc/init.d/redis_6379 start  服务启动
[root@master ~]# ps aux | grep redis           查看进程端口状态
[root@master ~]# tail -f /var/log/sentinel.log 查看日志文件

View the status of the database sentry on slave2

[root@slave2 ~]# redis-cli -h 192.168.1.11 -p 26379 info sentinel

It is found that after the original master server comes online, the master state will not switch to itself again. This is different from the vrrp service because vrrp has a priority setting, but the service here does not.