Redis学习笔记——主从复制，哨兵，集群

版权声明：转载请注明出处，谢谢 https://blog.csdn.net/m0_37867405/article/details/81911473

1. Redis脚本编写

1.Redis启动和关闭脚本

【start.sh】

#!/bin/sh
count=$#
if [ 1 -le $count ]; then
   for i in $*; do
   /usr/local/software/redis-4.0.10/src/redis-server $i;
   sleep 1;
   done;
else
 echo "[ERROR] params can not null"
fi

【stop.sh】

#!/bin/sh
# redis关闭脚本
PID=`ps -ef| grep "redis-server"| grep -v grep | awk '{ print $2 }'`;
echo $PID
if [ ! -z "$PID" ]; then
    for i in $PID; do
        kill -9 "$i";
        echo ""[INFO] kill redis pid is $i;
    done
       # kill -9 "$PID"
       # echo "[INFO] Kill is down."
else
    echo "[WARN] redis-server is not start."
fi
echo "[INFO] stop.sh  is complete."

2. Redis主从复制

1. 命令行主从

​分别启动三个Redis节点，使用客户端登录到从节点运行命令slaveof master_Host master_port

127.0.0.1:6379>slaveof 192.168.186.130 6379

2. 配置文件主从

1.主节点保持基本的配置即可

2.从节点配置增加：

主节点IP PORT

slaveof 192.168.186.130 6379

作用是让节点直接可以进行通信，

bind 0.0.0.0

3. Redis Sentinel

1. 配置文件

前提：哨兵模式建立在主从之上的模式，所以搭建哨兵之前需要创建redis主从复制

主从信息

# 主节点
[root@s130 redis-4.0.10]# ./src/redis-cli -h s130 -p 6379 info replication
# Replication
role:master
connected_slaves:2
slave0:ip=192.168.186.132,port=6379,state=online,offset=154,lag=1
slave1:ip=192.168.186.131,port=6379,state=online,offset=154,lag=0
master_replid:5a81d8459126d69689abb3da8f759fac8e3f37a7
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:154
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:154
# 从节点
[root@s131 redis-4.0.10]# ./src/redis-cli -h s131 -p 6379 info replication
# Replication
role:slave
master_host:192.168.186.130
master_port:6379
master_link_status:up
master_last_io_seconds_ago:10
master_sync_in_progress:0
slave_repl_offset:294
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:5a81d8459126d69689abb3da8f759fac8e3f37a7
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:294
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:57
repl_backlog_histlen:238

sentinel配置文件

​ 主节点和从节点配置一致

# 哨兵的默认端口
port 26379  
# 允许后端启动
daemonize yes 
# 日志文件的位置
logfile "26379.log"  
dir /opt/soft/redis/data  
# 监控的主节点信息，<master alias> <master host> <master port> <slave num>
sentinel monitor mymaster 192.168.186.130 6379 2
sentinel down-after-milliseconds mymaster 30000  
sentinel parallel-syncs mymaster 1  
sentinel failover-timeout mymaster 180000  

启动哨兵

# 方式一
redis-server redis-sentinel-26379.conf --sentinel
# 方式二
redis-sentinel redis-sentinel-26379.conf

sentinel查看状态

[root@s130 redis-4.0.10]# ./src/redis-cli -h s130 -p 26379 info sentinel
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
sentinel_simulate_failure_flags:0
master0:name=mymaster,status=ok,address=192.168.186.130:6379,slaves=2,sentinels=3

2. sentinel Api

[root@s130 redis-4.0.10]# ./src/redis-cli -h s130 -p 26379

s130:26379> sentinel masters
1)  1) "name"
    2) "mymaster"
    3) "ip"
    4) "192.168.186.130"
    5) "port"
    ...
# 类似api还有：
# 查看从节点信息
sentinel slaves <master name>
# 查看sentinels的信息
sentinel sentinels <master name>
# 获取直接点IP port
sentinel get-master-addr-by-name <master name>
# 对主节进行强制故障转移
sentinel failover <master name>
#
info sentinel
# 检查<quorum>
sentinel ckquorum
# 将sentinel 配置强制刷新到磁盘上,当配置文件文件损坏的时候，可以使用这个命令
sntinel flushconfig

4. Redis集群

1. 集群的搭建

集群的前提是一主一从，所以需要偶数个节点，所以第一次配置的时候出错了，这里三个节点不正确。

1.文件配置redis_cluster.conf

​ 三个节点，分别在三个不同的机器上面

logfile ""/usr/local/software/redis-4.0.10/selfdir/conf/cluster/logs/s130_cluster_6370.log"
port 6370
cluster-enabled yes
cluster-node-timeout 15000
cluster-config-file "/usr/local/software/redis-4.0.10/selfdir/conf/cluster/auto/s130-cluster_6370.conf"

2.安装ruby

遇到问题：

ERROR:  Error installing redis:
        redis requires Ruby version >= 2.2.2.

问题描述

ruby需要2.2.2以上

yum intall安装ruby只可以安装2.0.0的版本，而gem需要2.2.2以上的支持，所以这里需要使用RVM安装

解决方案

通过rvm来安装ruby

gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3
curl -sSL https://get.rvm.io | bash -s stable

find / -name rvm -print
source /usr/local/rvm/scripts/rvm
# 列出可安装的版本
rvm list known
rvm install 2.5.1
# 使用版本
rvm use 2.3.3
rvm use 2.3.3 --default
# 删除版本
rvm remove 2.0.0

或者直接编译源码

#  下载ruby
wget https:// cache.ruby-lang.org/pub/ruby/2.3/ruby-2.3.1.tar.gz
# 安装ruby
tar xvf ruby-2.3.1.tar.gz
./configure -prefix=/usr/local/ruby
make
make install
cd /usr/local/ruby
sudo cp bin/ruby /usr/local/bin
sudo cp bin/gem /usr/local/bin

3. 安装Redis接口

# 采用如下安装方式会安装4.0.2版本的redis.db接口，这里会出现集群搭建失败的这是官方的bug
# [ERR] Calling MIGRATE: ERR Syntax error, try CLIENT (LIST | KILL ...
gem install redis
# 解决方式
gem uninstall redis --version 4.0.2
# 降低版本
gem install redis --version 3.3.3

5. 创建集群

./src/redis-trib.rb create --replicas 1 192.168.186.130:6379 192.168.186.131:6379 192.168.186.132:6379

​ 创建集群失败

>>> Creating cluster
*** ERROR: Invalid configuration for cluster creation.
*** Redis Cluster requires at least 3 master nodes.
*** This is not possible with 3 nodes and 1 replicas per node.
*** At least 6 nodes are required.

​

​ 解决：创建六个节点

[root@s130 redis-4.0.10]# ./src/redis-trib.rb create --replicas 1 192.168.186.130:6379 192.168.186.131:6379 192.168.186.132:6379 192.168.186.130:6370 192.168.186.131:6370 192.168.186.132:6370
>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
192.168.186.130:6379
192.168.186.131:6379
192.168.186.132:6379
Adding replica 192.168.186.131:6370 to 192.168.186.130:6379
Adding replica 192.168.186.132:6370 to 192.168.186.131:6379
Adding replica 192.168.186.130:6370 to 192.168.186.132:6379
M: 4bf47f3999912052c66bbcb3373aec2740e83f56 192.168.186.130:6379
   slots:0-5460 (5461 slots) master
M: 5e8e62c0508575fdd217b77cd1ac1bfbe702e830 192.168.186.131:6379
   slots:5461-10922 (5462 slots) master
M: 9339564f30a9306d7d82451698b2ef8f0d92fad2 192.168.186.132:6379
   slots:10923-16383 (5461 slots) master
S: 004bb1639ecc875b0d10a59fe97afcc6d84fae39 192.168.186.130:6370
   replicates 9339564f30a9306d7d82451698b2ef8f0d92fad2
S: ff7e7a5d7d18802ef09c24baac1159e5626df8ad 192.168.186.131:6370
   replicates 4bf47f3999912052c66bbcb3373aec2740e83f56
S: b4fa4e4ef160a97659531b5b78b22b0af026deb2 192.168.186.132:6370
   replicates 5e8e62c0508575fdd217b77cd1ac1bfbe702e830
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join......
>>> Performing Cluster Check (using node 192.168.186.130:6379)
M: 4bf47f3999912052c66bbcb3373aec2740e83f56 192.168.186.130:6379
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
M: 9339564f30a9306d7d82451698b2ef8f0d92fad2 192.168.186.132:6379
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
M: 5e8e62c0508575fdd217b77cd1ac1bfbe702e830 192.168.186.131:6379
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
S: ff7e7a5d7d18802ef09c24baac1159e5626df8ad 192.168.186.131:6370
   slots: (0 slots) slave
   replicates 4bf47f3999912052c66bbcb3373aec2740e83f56
S: 004bb1639ecc875b0d10a59fe97afcc6d84fae39 192.168.186.130:6370
   slots: (0 slots) slave
   replicates 9339564f30a9306d7d82451698b2ef8f0d92fad2
S: b4fa4e4ef160a97659531b5b78b22b0af026deb2 192.168.186.132:6370
   slots: (0 slots) slave
   replicates 5e8e62c0508575fdd217b77cd1ac1bfbe702e830
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

6.集群API：

$>./src/redis-cli -h 192.168.186.130 -p 6379 cluster info

[root@s132 redis-4.0.10]# ./src/redis-cli -h 192.168.186.130 -p 6379 cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_ping_sent:224
cluster_stats_messages_pong_sent:222
cluster_stats_messages_sent:446
cluster_stats_messages_ping_received:217
cluster_stats_messages_pong_received:224
cluster_stats_messages_meet_received:5
cluster_stats_messages_received:446
# api还包括
cluster nodes
cluster slots

7. 集群健康检查：

redis-trib.rb check 127.0.0.1:6379
redis-trib.rb check 127.0.0.1:6481

8. 集群伸缩

Redis集群最大的特点就是满足集群伸缩

8.1 redis-trib.rb添加新节点：

# 命令格式
redis-trib.rb add-node new_host:new_port existing_host:existing_port --slave 
    --master-id <arg>
# 实操
./src/redis-trib.rb add-node 192.168.186.130:6371 192.168.186.130:6379
./src/redis-trib.rb add-node 192.168.186.131:6371 192.168.186.130:6379

数据迁移和槽迁移

使用Redis提供的工具进行迁移，我们连个节点192.168.186.130:6371作为主节点master, 192.168.186.131:6371作为从节点，不需要进行数据的迁移

# 命令格式
# host:port 现存集群中的任意节点，用于读取集群信息
# --from 制定源节点的id，可以是all，表示现存集群中的所有主节点  --to 目标节点6371，动态输入
# --slots 迁移槽的个数，动态输入
# --yes 当打印出reshard执行计划时，是否需要用户输入yes确认后再执行reshard。
# -- timeout 迁移超时时间
# --pipeline 批量迁移的个数
redis-trib.rb reshard host:port --from <arg> --to <arg> --slots <arg> --yes --timeout 
    <arg> --pipeline <arg>
# 迁移演示
$>./src/redis-trib.rb reshard 192.168.186.130:6379
    # 需要迁移多少个槽
    How many slots do you want to move (from 1 to 16384)? 4096
    # 目标节点ID
    What is the receiving node ID? 4f26250591522003f4058ac6d6343b46245f19e5
    # 源节点ID，多个使用“,”，分开,all代表所有主节点
    Source node #1:all

查看节点直接均衡度：

[root@s130 redis-4.0.10]# ./src/redis-trib.rb rebalance 192.168.186.130:6379
>>> Performing Cluster Check (using node 192.168.186.130:6379)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
*** No rebalancing needed! All nodes are within the 2.0% threshold.

为新的主节点添加从节点

# 语法./src/redis-cli -h <slaveIP> -p <slavePORT> cluster replicate <masterNodeID>
$>./src/redis-cli -h 192.168.186.131 -p 6371 cluster replicate 4f26250591522003f4058ac6d6343b46245f19e5
OK

8.2 集群收缩

收缩思路：

1. 将要剔除的节点数据槽（slots）转移到其他节点命令./src/redis-trib.rb reshard host:port
2. 为了平均分配，原先四个主节点，每个节点有16384/4=4096个节点，现在收缩为3个主节点，主节点槽数分别为：5461,5461,5362；所以，需要从剔除节点分别向其他接收节点转移1365,1365,1366个槽，运行三次下面命令即可

./src/redis-trib.rb reshard 192.168.186.130:6379

3.要剔除的节点槽数已经变为0，从集群中剔除节点

​ 这里有两个节点：master 以及该节点的备份节点：slave,备份节点不具有槽的特点，所以不需要进行槽转移

./src/redis-cli -h 192.168.186.130 -p 6379 cluster forget{down NodeId}
./src/redis-cli -h 192.168.186.130 -p 6379 cluster forget 4f26250591522003f4058ac6d6343b46245f19e5
./src/redis-cli -h 192.168.186.130 -p 6379 cluster forget 2e014b124bf786fb993493920f4c80ece4d7596c

实际生产环境可以使用命令替代cluster forget

redis-trib.rb del-node{host:port}{downNodeId}

8.3 java操作Redis集群

关闭保护模式

DENIED Redis is running in protected mode because protected mode is enabled, no bind address was specified, no authentication password is requested to clients. In this mode connections are only accepted from the loopback interface. If you want to connect from external computers to Redis you may adopt one of the following solutions: 1) Just disable protected mode sending the command ‘CONFIG SET protected-mode no’ from the loopback interface by connecting to Redis from the same host the server is running, however MAKE SURE Redis is not publicly accessible from internet if you do so. Use CONFIG REWRITE to make this change permanent. 2) Alternatively you can just disable the protected mode by editing the Redis configuration file, and setting the protected mode option to ‘no’, and then restarting the server. 3) If you started the server manually just for testing, restart it with the ‘–protected-mode no’ option. 4) Setup a bind address or an authentication password. NOTE: You only need to do one of the above things in order for the server to start accepting connections from the outside.

解决：bind 0.0.0.0