Redis主从集群及哨兵模式

本次实验环境准备用一台服务器模拟3台redis服务器,1主2从

主从集群搭建

第一步：安装Redis

安装Redis,参考前面安装Redis文章，保证单机使用没有问题。

第二步：配置服务器文件

定位到安装后的redis目录：cd /usr/local/redis

对单机的redis配置文件拷贝出3份出来

cp redis.conf  redis6381.conf

cp redis.conf  redis6382.conf

cp redis.conf  redis6383.conf

清空新拷贝的三份配置文件，命令为 “> 文件名”

[root@localhost redis]# > redis6381.conf

[root@localhost redis]# > redis6382.conf

[root@localhost redis]# > redis6383.conf

再对三份配置文件分别配置内容

6381端口的配置文件

include /usr/local/redis/redis.conf  #包含redis.conf文件
daemonize yes
port 6381
pidfile /var/run/redis_6381.pid
logfile 6381.log
dbfilename dump6381.rdb

6382端口的配置文件

include /usr/local/redis/redis.conf  #包含redis.conf文件
daemonize yes
port 6382
pidfile /var/run/redis_6382.pid
logfile 6382.log
dbfilename dump6382.rdb
slaveof 127.0.0.1 6381    #说明是从属于6381端口的主机

6383端口的配置文件

include /usr/local/redis/redis.conf  #包含redis.conf文件
daemonize yes
port 6383
pidfile /var/run/redis_6383.pid
logfile 6383.log
dbfilename dump6383.rdb
slaveof 127.0.0.1 6381  #说明是从属于6381端口的主机

第三步：启动redis服务器

模拟启动三台redis服务器

[root@localhost redis]# ./bin/redis-server redis6381.conf
[root@localhost redis]# ./bin/redis-server redis6382.conf
[root@localhost redis]# ./bin/redis-server redis6383.conf

查看进程

[root@localhost redis]# ps -ef|grep redis
root      90648      1  0 17:17 ?        00:00:00 ./bin/redis-server 127.0.0.1:6381
root      90652      1  0 17:17 ?        00:00:00 ./bin/redis-server 127.0.0.1:6382
root      90657      1  1 17:17 ?        00:00:00 ./bin/redis-server 127.0.0.1:6383
root      90664   4536  0 17:17 pts/2    00:00:00 grep --color=auto redis

说明3台服务器已经启动成功，端口分别是指定的6381,6382,6383

第四步，客户端连接redis服务器

分别打开三个窗口，定位到redis目录：cd /usr/local/redis

通过客户端工具连接不同端口的服务器

如连接6381端口服务器

[root@localhost redis]# ./bin/redis-cli -p 6381

查看信息，如下：

127.0.0.1:6381> info replication

# Replication
role:master
connected_slaves:2
slave0:ip=127.0.0.1,port=6382,state=online,offset=253,lag=1
slave1:ip=127.0.0.1,port=6383,state=online,offset=253,lag=1
master_repl_offset:253
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:252

可以看到它是主服务器，我们在前面配置文件中指定它就是主服务器。

因为是主机，它可读可写，写的数据同步到从机。

写：

127.0.0.1:6381> set key1 v1
OK
127.0.0.1:6381> set key2 v2
OK

读:

127.0.0.1:6381> get key1
"v1"
127.0.0.1:6381> get key2
"v2"

如连接6382端口服务器
[root@localhost redis]# ./bin/redis-cli -p 6382

 查看信息，如下：

127.0.0.1:6382> info replication
# Replication
role:slave
master_host:127.0.0.1
master_port:6381
master_link_status:up
master_last_io_seconds_ago:10
master_sync_in_progress:0
slave_repl_offset:883
slave_priority:100
slave_read_only:1
connected_slaves:0
master_repl_offset:0
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0

可以看到，它是从服务器，我们在前面配置文件中指定它就是从服务器。

因为是从机，它只读不可写。数据只能从主机6381同步。

读：

127.0.0.1:6382> keys *
1) "key1"
2) "key2"
127.0.0.1:6382> get key2
"v2"

写：

127.0.0.1:6382> set key3 v3
(error) READONLY You can't write against a read only slave.

哨兵模式

配置哨兵配置文件

定为到redis的源文件目录，将哨兵配置文件sentinel.conf作为模板，拷贝出3份

[root@localhost local]# cd redis-3.2.9/

[root@localhost redis-3.2.9]# cp sentinel.conf  sentinel26381.conf
[root@localhost redis-3.2.9]# cp sentinel.conf  sentinel26382.conf
[root@localhost redis-3.2.9]# cp sentinel.conf  sentinel26383.conf

分别修改拷贝的配置文件

[root@localhost redis-3.2.9]# vim sentinel26381.conf

将哨兵的工作端口端口26379和监控的redis主机127.0.0.1 6379 2修改为自己定义的值

模板中的两处要被修改的地方：

port 26379

sentinel monitor mymaster 127.0.0.1 6379 2

我这里修改为该哨兵端口为26381，监控的redis主机为127.0.0.1 6381

port 26381

sentinel monitor mymaster 127.0.0.1 6381 2

其他两台哨兵配置文件也按照这样修改，只是哨兵的端口要不一样，监控的redis主机应该一样。

启动哨兵程序

分别打开3个窗口，定为到/usr/local/redis-3.2.9/src，找到redis-sentinel，配合上步的配置文件启动

注意需要3个窗口，因为这里是前台启动，需要看监控信息

[root@localhost src]# ./redis-sentinel ../sentinel26381.conf

[root@localhost src]# ./redis-sentinel ../sentinel26382.conf

[root@localhost src]# ./redis-sentinel ../sentinel26383.conf

等3台哨兵都启动成功以后，

分别查看监控信息，26381哨兵的监控信息，如下：

[root@localhost src]# ./redis-sentinel ../sentinel26381.conf
94744:X 01 Feb 18:29:03.171 * Increased maximum number of open files to 10032 (it was originally set to 1024).
                _._                                                  
           _.-``__ ''-._                                             
      _.-``    `.  `_.  ''-._           Redis 3.2.9 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._                                   
 (    '      ,       .-`  | `,    )     Running in sentinel mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 26381
 |    `-._   `._    /     _.-'    |     PID: 94744
  `-._    `-._  `-./  _.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |           http://redis.io        
  `-._    `-._`-.__.-'_.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |                                  
  `-._    `-._`-.__.-'_.-'    _.-'                                   
      `-._    `-.__.-'    _.-'                                       
          `-._        _.-'                                           
              `-.__.-'                                               

94744:X 01 Feb 18:29:03.172 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
94744:X 01 Feb 18:29:03.174 # Sentinel ID is 9044885c3cd355b168a93bc68ed4c2b7c78ebca6
94744:X 01 Feb 18:29:03.174 # +monitor master mymaster 127.0.0.1 6381 quorum 2
94744:X 01 Feb 18:29:03.175 * +slave slave 127.0.0.1:6382 127.0.0.1 6382 @ mymaster 127.0.0.1 6381
94744:X 01 Feb 18:29:03.176 * +slave slave 127.0.0.1:6383 127.0.0.1 6383 @ mymaster 127.0.0.1 6381
/usr/local/redis-3.2.9/src/usr/local/redis-3.2.9/src94744:X 01 Feb 18:33:35.645 * +sentinel sentinel 7bc01d8db6cf93c5d1f9d0a146c9967c0c4b339b 127.0.0.1 26382 @ mymaster 127.0.0.1 6381
94744:X 01 Feb 18:34:08.801 * +sentinel sentinel 574de03c2a3e5ace4798af0df4945961398efdf1 127.0.0.1 26383 @ mymaster 127.0.0.1 6381

26382哨兵的监控信息，如下：

[root@localhost src]# ./redis-sentinel ../sentinel26382.conf
95009:X 01 Feb 18:33:33.626 * Increased maximum number of open files to 10032 (it was originally set to 1024).
                _._                                                  
           _.-``__ ''-._                                             
      _.-``    `.  `_.  ''-._           Redis 3.2.9 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._                                   
 (    '      ,       .-`  | `,    )     Running in sentinel mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 26382
 |    `-._   `._    /     _.-'    |     PID: 95009
  `-._    `-._  `-./  _.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |           http://redis.io        
  `-._    `-._`-.__.-'_.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |                                  
  `-._    `-._`-.__.-'_.-'    _.-'                                   
      `-._    `-.__.-'    _.-'                                       
          `-._        _.-'                                           
              `-.__.-'                                               

95009:X 01 Feb 18:33:33.627 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
95009:X 01 Feb 18:33:33.648 # Sentinel ID is 7bc01d8db6cf93c5d1f9d0a146c9967c0c4b339b
95009:X 01 Feb 18:33:33.648 # +monitor master mymaster 127.0.0.1 6381 quorum 2
95009:X 01 Feb 18:33:33.649 * +slave slave 127.0.0.1:6382 127.0.0.1 6382 @ mymaster 127.0.0.1 6381
95009:X 01 Feb 18:33:33.651 * +slave slave 127.0.0.1:6383 127.0.0.1 6383 @ mymaster 127.0.0.1 6381
95009:X 01 Feb 18:33:34.651 * +sentinel sentinel 9044885c3cd355b168a93bc68ed4c2b7c78ebca6 127.0.0.1 26381 @ mymaster 127.0.0.1 6381
95009:X 01 Feb 18:34:08.801 * +sentinel sentinel 574de03c2a3e5ace4798af0df4945961398efdf1 127.0.0.1 26383 @ mymaster 127.0.0.1 6381

26383哨兵的监控信息，如下：

[root@localhost src]# ./redis-sentinel ../sentinel26383.conf
95097:X 01 Feb 18:34:06.716 * Increased maximum number of open files to 10032 (it was originally set to 1024).
                _._                                                  
           _.-``__ ''-._                                             
      _.-``    `.  `_.  ''-._           Redis 3.2.9 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._                                   
 (    '      ,       .-`  | `,    )     Running in sentinel mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 26383
 |    `-._   `._    /     _.-'    |     PID: 95097
  `-._    `-._  `-./  _.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |           http://redis.io        
  `-._    `-._`-.__.-'_.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |                                  
  `-._    `-._`-.__.-'_.-'    _.-'                                   
      `-._    `-.__.-'    _.-'                                       
          `-._        _.-'                                           
              `-.__.-'                                               

95097:X 01 Feb 18:34:06.717 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
95097:X 01 Feb 18:34:06.719 # Sentinel ID is 574de03c2a3e5ace4798af0df4945961398efdf1
95097:X 01 Feb 18:34:06.719 # +monitor master mymaster 127.0.0.1 6381 quorum 2
95097:X 01 Feb 18:34:06.719 * +slave slave 127.0.0.1:6382 127.0.0.1 6382 @ mymaster 127.0.0.1 6381
95097:X 01 Feb 18:34:06.721 * +slave slave 127.0.0.1:6383 127.0.0.1 6383 @ mymaster 127.0.0.1 6381
95097:X 01 Feb 18:34:07.784 * +sentinel sentinel 9044885c3cd355b168a93bc68ed4c2b7c78ebca6 127.0.0.1 26381 @ mymaster 127.0.0.1 6381
95097:X 01 Feb 18:34:08.425 * +sentinel sentinel 7bc01d8db6cf93c5d1f9d0a146c9967c0c4b339b 127.0.0.1 26382 @ mymaster 127.0.0.1 6381

 模拟故障，让主从自动切换

主动让redis的主服务器6381关掉，模拟主机挂机的状况

127.0.0.1:6381> shutdown

观看哨兵服务器，会发现以下信息

[root@localhost src]# ./redis-sentinel ../sentinel26382.conf
95009:X 01 Feb 18:33:33.626 * Increased maximum number of open files to 10032 (it was originally set to 1024).
                _._                                                  
           _.-``__ ''-._                                             
      _.-``    `.  `_.  ''-._           Redis 3.2.9 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._                                   
 (    '      ,       .-`  | `,    )     Running in sentinel mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 26382
 |    `-._   `._    /     _.-'    |     PID: 95009
  `-._    `-._  `-./  _.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |           http://redis.io        
  `-._    `-._`-.__.-'_.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |                                  
  `-._    `-._`-.__.-'_.-'    _.-'                                   
      `-._    `-.__.-'    _.-'                                       
          `-._        _.-'                                           
              `-.__.-'                                               

95009:X 01 Feb 18:33:33.627 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
95009:X 01 Feb 18:33:33.648 # Sentinel ID is 7bc01d8db6cf93c5d1f9d0a146c9967c0c4b339b
95009:X 01 Feb 18:33:33.648 # +monitor master mymaster 127.0.0.1 6381 quorum 2
95009:X 01 Feb 18:33:33.649 * +slave slave 127.0.0.1:6382 127.0.0.1 6382 @ mymaster 127.0.0.1 6381
95009:X 01 Feb 18:33:33.651 * +slave slave 127.0.0.1:6383 127.0.0.1 6383 @ mymaster 127.0.0.1 6381
95009:X 01 Feb 18:33:34.651 * +sentinel sentinel 9044885c3cd355b168a93bc68ed4c2b7c78ebca6 127.0.0.1 26381 @ mymaster 127.0.0.1 6381
95009:X 01 Feb 18:34:08.801 * +sentinel sentinel 574de03c2a3e5ace4798af0df4945961398efdf1 127.0.0.1 26383 @ mymaster 127.0.0.1 6381
95009:X 01 Feb 18:59:42.232 # +sdown master mymaster 127.0.0.1 6381
95009:X 01 Feb 18:59:42.316 # +odown master mymaster 127.0.0.1 6381 #quorum 2/2
95009:X 01 Feb 18:59:42.316 # +new-epoch 1
95009:X 01 Feb 18:59:42.316 # +try-failover master mymaster 127.0.0.1 6381
95009:X 01 Feb 18:59:42.325 # +vote-for-leader 7bc01d8db6cf93c5d1f9d0a146c9967c0c4b339b 1
95009:X 01 Feb 18:59:42.363 # 9044885c3cd355b168a93bc68ed4c2b7c78ebca6 voted for 7bc01d8db6cf93c5d1f9d0a146c9967c0c4b339b 1
95009:X 01 Feb 18:59:42.363 # 574de03c2a3e5ace4798af0df4945961398efdf1 voted for 7bc01d8db6cf93c5d1f9d0a146c9967c0c4b339b 1
95009:X 01 Feb 18:59:42.379 # +elected-leader master mymaster 127.0.0.1 6381
95009:X 01 Feb 18:59:42.379 # +failover-state-select-slave master mymaster 127.0.0.1 6381
95009:X 01 Feb 18:59:42.470 # +selected-slave slave 127.0.0.1:6383 127.0.0.1 6383 @ mymaster 127.0.0.1 6381
95009:X 01 Feb 18:59:42.470 * +failover-state-send-slaveof-noone slave 127.0.0.1:6383 127.0.0.1 6383 @ mymaster 127.0.0.1 6381
95009:X 01 Feb 18:59:42.522 * +failover-state-wait-promotion slave 127.0.0.1:6383 127.0.0.1 6383 @ mymaster 127.0.0.1 6381
95009:X 01 Feb 18:59:43.315 # +promoted-slave slave 127.0.0.1:6383 127.0.0.1 6383 @ mymaster 127.0.0.1 6381
95009:X 01 Feb 18:59:43.315 # +failover-state-reconf-slaves master mymaster 127.0.0.1 6381
95009:X 01 Feb 18:59:43.376 * +slave-reconf-sent slave 127.0.0.1:6382 127.0.0.1 6382 @ mymaster 127.0.0.1 6381
95009:X 01 Feb 18:59:44.363 * +slave-reconf-inprog slave 127.0.0.1:6382 127.0.0.1 6382 @ mymaster 127.0.0.1 6381
95009:X 01 Feb 18:59:44.363 * +slave-reconf-done slave 127.0.0.1:6382 127.0.0.1 6382 @ mymaster 127.0.0.1 6381
95009:X 01 Feb 18:59:44.422 # +failover-end master mymaster 127.0.0.1 6381
95009:X 01 Feb 18:59:44.422 # +switch-master mymaster 127.0.0.1 6381 127.0.0.1 6383
95009:X 01 Feb 18:59:44.422 * +slave slave 127.0.0.1:6382 127.0.0.1 6382 @ mymaster 127.0.0.1 6383
95009:X 01 Feb 18:59:44.422 * +slave slave 127.0.0.1:6381 127.0.0.1 6381 @ mymaster 127.0.0.1 6383
95009:X 01 Feb 19:00:14.465 # +sdown slave 127.0.0.1:6381 127.0.0.1 6381 @ mymaster 127.0.0.1 6383

说明集群的主服务器已经切换到了6383上，即原来的从机6383变为了主机

我们去看一下6383的信息

127.0.0.1:6383>  info replication
# Replication
role:master
connected_slaves:1
slave0:ip=127.0.0.1,port=6382,state=online,offset=19848,lag=1
master_repl_offset:19981
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:19980

确实角色为master,它下面挂载一台从机6382

它现在升级为主机，所以可以读写，验证如下：

127.0.0.1:6383> set key5 v5
OK
127.0.0.1:6383> get key5
"v5"

再去看下一6382从机的信息

127.0.0.1:6382> info replication
# Replication
role:slave
master_host:127.0.0.1
master_port:6383
master_link_status:up
master_last_io_seconds_ago:0
master_sync_in_progress:0
slave_repl_offset:17440
slave_priority:100
slave_read_only:1
connected_slaves:0
master_repl_offset:0
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0

说明它确实从属的主机为6383

它还是只能读，数据只能从主机同步

127.0.0.1:6382> set key6 v6
(error) READONLY You can't write against a read only slave.
127.0.0.1:6382> get key5
"v5"

假设原来的6381服务器已经恢复正常了，我们来启动它

可以看到，6381启动后，自动变为了从机，主机还是后来的6383，说明原来的主机恢复后，不会恢复它原来的地位。

因为它已经是从机，所以只能读，不能写

再看主服务器6383的信息，它已经有两个从机了，6381是后来恢复后自动挂载上的

哨兵服务器26381上看到对应的自动挂载信息

哨兵服务器26382上看到对应的自动挂载信息

哨兵服务器26383上看到对应的自动挂载信息