一、下载

https://redis.io/download

http://download.redis.io/releases/

二、说明

redis5.0版本之后可以直接使用redis-cli命令创建集群，不使用redis-trib.rb命令了。

本文一在主机上建立3主3从集群。实际最少需要3台服务器，6个节点。当然6个服务器更好了。

rdb和aof可以同时开启。如果只做缓存，可以不开启aof。

Redis 集群的数据分片

Redis 集群没有使用一致性hash, 而是引入了 哈希槽的概念.

Redis 集群有16384个哈希槽,每个key通过CRC16校验后对16384取模来决定放置哪个槽.集群的每个节点负责一部分hash槽,举个例子,比如当前集群有3个节点,那么:

节点 A 包含 0 到 5500号哈希槽.
节点 B 包含5501 到 11000 号哈希槽.
节点 C 包含11001 到 16384号哈希槽.

这种结构很容易添加或者删除节点. 比如如果我想新添加个节点D, 我需要从节点 A, B, C中得部分槽到D上. 如果我想移除节点A,需要将A中的槽移到B和C节点上,然后将没有任何槽的A节点从集群中移除即可. 由于从一个节点将哈希槽移动到另一个节点并不会停止服务,所以无论添加删除或者改变某个节点的哈希槽的数量都不会造成集群不可用的状态.

Redis 集群的目标

Redis 集群是 Redis 的一个分布式实现，主要是为了实现以下这些目标（按在设计中的重要性排序）：

在1000个节点的时候仍能表现得很好并且可扩展性（scalability）是线性的。
没有合并操作，这样在 Redis 的数据模型中最典型的大数据值中也能有很好的表现。
写入安全（Write safety）：那些与大多数节点相连的客户端所做的写入操作，系统尝试全部都保存下来。不过公认的，还是会有小部分（small windows?）写入会丢失。
可用性（Availability）：在绝大多数的主节点（master node）是可达的，并且对于每一个不可达的主节点都至少有一个它的从节点（slave）可达的情况下，Redis 集群仍能进行分区（partitions）操作。

这篇文档要讲的是，在 Redis 仓库（放在Github上）中的 unstable 分支中实现的功能。

实现的功能子集

Redis 集群实现了所有在非分布式 Redis 版本中出现的处理单一键值（key）的命令。那些使用多个键值的复杂操作，比如 set 里的并集（unions）和交集（intersections）操作，就没有实现。通常来说，那些处理命令的节点获取不到键值的所有操作都不会被实现。在将来，用户或许可以通过使用 MIGRATE COPY 命令，在集群上用计算节点（Computation Nodes）来执行多键值的只读操作，但 Redis 集群本身不会执行复杂的多键值操作来把键值在节点间移来移去。 Redis 集群不像单机版本的 Redis 那样支持多个数据库，集群只有数据库 0，而且也不支持 SELECT 命令。

三、安装

1、上传解压文件

上传redis-5.0.8.tar.gz到/opt下
[root@dev1 opt]# tar -xvf redis-5.0.8.tar.gz
[root@dev1 opt]# cd redis-5.0.8/

2、make，make install，make test
[root@dev1 redis-5.0.8]# make
[root@dev1 redis-5.0.8]# make PREFIX=/opt/redis5-cluster install

[root@dev1 redis-5.0.8]# make
-bash: /usr/bin/make: 没有那个文件或目录
[root@dev1 redis-5.0.8]# yum intall make #安装make
[root@dev1 redis-5.0.8]# make
……
make[3]：gcc：命令未找到
……
[root@dev1 redis-5.0.8]# yum install gcc #安装gcc

重新解压代码make
[root@dev1 redis-5.0.8]# make

[root@dev1 redis-5.0.8]# make PREFIX=/opt/redis5-cluster install #指定安装目录PREFIX

[root@dev1 redis-5.0.8]# make test
cd src && make test
make[1]: 进入目录“/opt/redis-5.0.8/src”
CC Makefile.dep
You need tcl 8.5 or newer in order to run the Redis test

[root@dev1 redis-5.0.8]# yum install tcl #安装tcl

[root@dev1 redis-5.0.8]# make test
ERROR:
*** [err]: Cant' start the Redis server
ERROR:
*** [err]: WAIT should not acknowledge 1 additional copy if slave is blocked in tests/unit/wait.tcl
Expected condition '[$master wait 1 3000] == 0' to be true ([::redis::redisHandle9 wait 1 3000] == 0)
*** [err]: pending querybuf: check size of pending_querybuf after set a big value in tests/unit/pendingquerybuf.tcl
the used_memory of replica is much larger than master. Master:43869464 Replica:60643200
Cleanup: may take some time... OK
make[1]: *** [Makefile:271：test] 错误 1
make[1]: 离开目录“/opt/redis-5.0.8/src”
make: *** [Makefile:6：test] 错误 2

[root@dev1 redis-5.0.8]# ps -ef|grep redis
[root@dev1 redis-5.0.8]# kill -9 进程号

[root@dev1 redis-5.0.8]# make test
\o/ All tests passed without errors!

Cleanup: may take some time... OK
make[1]: 离开目录“/opt/redis-5.0.8/src”

四、配置单节点

[root@dev1 redis-5.0.8]# cd /opt/redis5-cluster/
[root@dev1 redis5-cluster]# ll
总用量 0
drwxr-xr-x. 2 root root 150 3月 26 04:11 bin
[root@dev1 redis5-cluster]# cd bin
[root@dev1 bin]# ll
总用量 42532
-rwxr-xr-x. 1 root root 5945112 3月 26 04:06 redis-benchmark
-rwxr-xr-x. 1 root root 10378640 3月 26 04:06 redis-check-aof
-rwxr-xr-x. 1 root root 10378640 3月 26 04:06 redis-check-rdb
-rwxr-xr-x. 1 root root 6466256 3月 26 04:06 redis-cli
lrwxrwxrwx. 1 root root 12 3月 26 04:06 redis-sentinel -> redis-server
-rwxr-xr-x. 1 root root 10378640 3月 26 04:06 redis-server

4.1创建redis实例目录

[root@dev1 redis5-cluster]# cd ..
[root@dev1 redis5-cluster]# mkdir 7001 7002 7003 7004 7005 7006
[root@dev1 redis5-cluster]# ll
总用量 0
drwxr-xr-x. 2 root root 6 3月 26 14:13 7001
drwxr-xr-x. 2 root root 6 3月 26 14:13 7002
drwxr-xr-x. 2 root root 6 3月 26 14:13 7003
drwxr-xr-x. 2 root root 6 3月 26 14:13 7004
drwxr-xr-x. 2 root root 6 3月 26 14:13 7005
drwxr-xr-x. 2 root root 6 3月 26 14:13 7006
drwxr-xr-x. 2 root root 134 3月 26 13:22 bin

4.2拷贝编辑配置文件

[root@dev1 redis5-cluster]# cp /opt/redis-5.0.8/redis.conf /opt/redis5-cluster/7001/redis.conf
[root@dev1 redis5-cluster]# cd 7001
[root@dev1 7001]# ll
总用量 64
-rw-r--r--. 1 root root 61797 3月 26 14:16 redis.conf

编辑配置文件

#关闭保护模式
#protected-mode yes
protected-mode no

#设置端口号
#port 6379
port 7001

#守护进程，在后台运行
#daemonize no
daemonize yes

#设置进程ID文件路径
#pidfile /var/run/redis_6379.pid
pidfile /opt/redis5-cluster/7001/redis_7001.pid

#设置rdb文件名（不用改）
dbfilename dump.rdb
设置rdb文件（及aof文件）路径
#dir ./
dir /opt/redis5-cluster/7001

#设置日志文件路径
#logfile ""
logfile /opt/redis5-cluster/7001/redis.log

#设置连接密码
# requirepass foobared
requirepass redis508

#设置主密码（主从复制时使用）
# masterauth <master-password>
masterauth redis508

#设置最大内存 (先不设置)
# maxmemory <bytes>
#maxmemory 2gb

#开启AOF持久化（做缓存，为了提高效率，这个可以不开）
#appendonly no
appendonly yes

#设置AOF文件名称（不用改）
appendfilename "appendonly.aof"

#启用集群模式
cluster-enabled yes

#集群配置文件（自动创建）
# cluster-config-file nodes-6379.conf
cluster-config-file /opt/redis5-cluster/7001/nodes-7001.conf

#超时时间
# cluster-node-timeout 15000
cluster-node-timeout 5000

4.3启动redis服务
[root@dev1 7001]# /opt/redis5-cluster/bin/redis-server /opt/redis5-cluster/7001/redis.conf
[root@dev1 7001]# ps -ef|grep redis
root 77372 1 0 14:41 ? 00:00:00 /opt/redis5-cluster/bin/redis-server 127.0.0.1:7001
root 77379 73631 0 14:42 pts/5 00:00:00 grep --color=auto redis

[root@dev1 7001]# ll
总用量 72
-rw-r--r--. 1 root root 61982 3月 26 14:40 redis.conf
-rw-r--r--. 1 root root 2496 3月 26 14:41 redis.log
-rw-r--r--. 1 root root 6 3月 26 14:41 redis.pid
这会rdb还没保存

4.4查看日志

可以打开另一个窗口查看日志
[root@dev1 7001]# tail -f redis.log
77601:C 26 Mar 2020 15:00:30.324 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
77601:C 26 Mar 2020 15:00:30.324 # Redis version=5.0.8, bits=64, commit=00000000, modified=0, pid=77601, just started
77601:C 26 Mar 2020 15:00:30.324 # Configuration loaded
77602:M 26 Mar 2020 15:00:30.326 * Increased maximum number of open files to 10032 (it was originally set to 1024).
_._
_.-``__ ''-._
_.-`` `. `_. ''-._ Redis 5.0.8 (00000000/0) 64 bit
.-`` .-```. ```\/ _.,_ ''-._
( ' , .-` | `, ) Running in standalone mode
|`-._`-...-` __...-.``-._|'` _.-'| Port: 7001
| `-._ `._ / _.-' | PID: 77602
`-._ `-._ `-./ _.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' | http://redis.io
`-._ `-._`-.__.-'_.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' |
`-._ `-._`-.__.-'_.-' _.-'
`-._ `-.__.-' _.-'
`-._ _.-'
`-.__.-'

77602:M 26 Mar 2020 15:00:30.327 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
77602:M 26 Mar 2020 15:00:30.327 # Server initialized
77602:M 26 Mar 2020 15:00:30.327 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
77602:M 26 Mar 2020 15:00:30.327 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
77602:M 26 Mar 2020 15:00:30.327 * Ready to accept connections

4.5解决3个警告
警告：无法强制执行TCP backlog设置511，因为/proc/sys/net/core/somaxconn被设置为较低的值128。
redis优化/proc/sys/net/core/somaxconn
https://blog.csdn.net/haveqing/article/details/84835397
警告overcommit_memory设置为0！在内存不足的情况下，后台保存可能会失败。若要解决此问题，请将“vm.overcommit_memory=1”添加到/etc/sysctl.conf，然后重新启动或运行命令“sysctl vm.overcommit_memory=1”以使其生效。
警告您在内核中启用了透明大页（THP）支持。这将在Redis中造成延迟和内存使用问题。要解决此问题，请以root用户身份运行命令“echo never > /sys/kernel/mm/transparent_hugepage/enabled”，并将其添加到您的/etc/rc.local中，以便在重启后保留设置。禁用THP后，必须重新启动Redis。

重启redis后就没警告了

4.6测试单节点
[root@dev1 7001]# /opt/redis5-cluster/bin/redis-cli -h 127.0.0.1 -p 7001 -a redis508
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
127.0.0.1:7001> ping
PONG
127.0.0.1:7001> set foo bar
(error) CLUSTERDOWN Hash slot not served

4.7停止redis
[root@dev1 7001]# /opt/redis5-cluster/bin/redis-cli -h 127.0.0.1 -p 7001 -a redis508 shutdown
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
[root@dev1 7001]# /opt/redis5-cluster/bin/redis-cli -h 127.0.0.1 -p 7001
127.0.0.1:7001> auth redis508
OK
127.0.0.1:7001> shutdown
not connected> exit

4.8配置6个节点，并启动
把7001下的redis.conf复制出5份，并修改相应端口号，

启动6个节点
[root@dev1 7001]# /opt/redis5-cluster/bin/redis-server /opt/redis5-cluster/7001/redis.conf
[root@dev1 7001]# /opt/redis5-cluster/bin/redis-server /opt/redis5-cluster/7002/redis.conf
[root@dev1 7001]# /opt/redis5-cluster/bin/redis-server /opt/redis5-cluster/7003/redis.conf
[root@dev1 7001]# /opt/redis5-cluster/bin/redis-server /opt/redis5-cluster/7004/redis.conf
[root@dev1 7001]# /opt/redis5-cluster/bin/redis-server /opt/redis5-cluster/7005/redis.conf
[root@dev1 7001]# /opt/redis5-cluster/bin/redis-server /opt/redis5-cluster/7006/redis.conf

查看进程
[root@dev1 redis5-cluster]# ps -ef|grep redis
root 77576 76538 0 3月26 pts/2 00:00:00 tail -f redis.log
root 83575 1 0 00:03 ? 00:00:00 /opt/redis5-cluster/bin/redis-server 127.0.0.1:7001 [cluster]
root 83580 1 0 00:03 ? 00:00:00 /opt/redis5-cluster/bin/redis-server 127.0.0.1:7002 [cluster]
root 83585 1 0 00:03 ? 00:00:00 /opt/redis5-cluster/bin/redis-server 127.0.0.1:7003 [cluster]
root 83590 1 0 00:03 ? 00:00:00 /opt/redis5-cluster/bin/redis-server 127.0.0.1:7004 [cluster]
root 83595 1 0 00:04 ? 00:00:00 /opt/redis5-cluster/bin/redis-server 127.0.0.1:7005 [cluster]
root 83600 1 0 00:04 ? 00:00:00 /opt/redis5-cluster/bin/redis-server 127.0.0.1:7006 [cluster]
root 83626 73631 0 00:06 pts/5 00:00:00 grep --color=auto redis

五、创建集群

关键的就是这么一句

# --cluster-replicas 1 表示主从配置比，1表示的是1:1，前三个是主，后三个是从
# 若配置文件中设置的密码，则还需要加上-a passwod

[root@dev1 redis5-cluster]# /opt/redis5-cluster/bin/redis-cli --cluster create 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 127.0.0.1:7006 --cluster-replicas 1 -a redis508
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 127.0.0.1:7005 to 127.0.0.1:7001
Adding replica 127.0.0.1:7006 to 127.0.0.1:7002
Adding replica 127.0.0.1:7004 to 127.0.0.1:7003
>>> Trying to optimize slaves allocation for anti-affinity
[WARNING] Some slaves are in the same host as their master
M: 1d082cde3e2cd0a105051fee0bfc54068075f6f6 127.0.0.1:7001
slots:[0-5460] (5461 slots) master
M: 64498cc1ff0492cccfc7f7a38f84a6f9bc9c030f 127.0.0.1:7002
slots:[5461-10922] (5462 slots) master
M: 40b51c0777d77593a24f2cbc821572373bd67de8 127.0.0.1:7003
slots:[10923-16383] (5461 slots) master
S: 74932e253e75529d7a07a6a4d24f87be588906a3 127.0.0.1:7004
replicates 64498cc1ff0492cccfc7f7a38f84a6f9bc9c030f
S: 309d52444efb383853c2a4c0c4241b6f18677c07 127.0.0.1:7005
replicates 40b51c0777d77593a24f2cbc821572373bd67de8
S: 258394ebcbdbbd9097c95177b3abbdb972155f6a 127.0.0.1:7006
replicates 1d082cde3e2cd0a105051fee0bfc54068075f6f6
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
...
>>> Performing Cluster Check (using node 127.0.0.1:7001)
M: 1d082cde3e2cd0a105051fee0bfc54068075f6f6 127.0.0.1:7001
slots:[0-5460] (5461 slots) master
1 additional replica(s)
S: 74932e253e75529d7a07a6a4d24f87be588906a3 127.0.0.1:7004
slots: (0 slots) slave
replicates 64498cc1ff0492cccfc7f7a38f84a6f9bc9c030f
S: 309d52444efb383853c2a4c0c4241b6f18677c07 127.0.0.1:7005
slots: (0 slots) slave
replicates 40b51c0777d77593a24f2cbc821572373bd67de8
M: 40b51c0777d77593a24f2cbc821572373bd67de8 127.0.0.1:7003
slots:[10923-16383] (5461 slots) master
1 additional replica(s)
S: 258394ebcbdbbd9097c95177b3abbdb972155f6a 127.0.0.1:7006
slots: (0 slots) slave
replicates 1d082cde3e2cd0a105051fee0bfc54068075f6f6
M: 64498cc1ff0492cccfc7f7a38f84a6f9bc9c030f 127.0.0.1:7002
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

5.1测试

普通连接到单节点可能会报错
[root@dev1 redis5-cluster]# /opt/redis5-cluster/bin/redis-cli -h 127.0.0.1 -p 7001 -a redis508
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
127.0.0.1:7001> set foo bar
(error) MOVED 12182 127.0.0.1:7003

使用-c连接集群节点
[root@dev1 redis5-cluster]# /opt/redis5-cluster/bin/redis-cli -c -h 127.0.0.1 -p 7001 -a redis508
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
127.0.0.1:7001> set foo bar
-> Redirected to slot [12182] located at 127.0.0.1:7003
OK
127.0.0.1:7003> get foo
"bar"

5.2查看集群节点
[root@dev1 ~]# /opt/redis5-cluster/bin/redis-cli -c -h 127.0.0.1 -p 7001 -a redis508 cluster nodes
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
74932e253e75529d7a07a6a4d24f87be588906a3 127.0.0.1:7004@17004 slave 64498cc1ff0492cccfc7f7a38f84a6f9bc9c030f 0 1585243280000 4 connected
309d52444efb383853c2a4c0c4241b6f18677c07 127.0.0.1:7005@17005 slave 40b51c0777d77593a24f2cbc821572373bd67de8 0 1585243279000 5 connected
40b51c0777d77593a24f2cbc821572373bd67de8 127.0.0.1:7003@17003 master - 0 1585243280000 3 connected 10923-16383
1d082cde3e2cd0a105051fee0bfc54068075f6f6 127.0.0.1:7001@17001 myself,master - 0 1585243280000 1 connected 0-5460
258394ebcbdbbd9097c95177b3abbdb972155f6a 127.0.0.1:7006@17006 slave 1d082cde3e2cd0a105051fee0bfc54068075f6f6 0 1585243279309 6 connected
64498cc1ff0492cccfc7f7a38f84a6f9bc9c030f 127.0.0.1:7002@17002 master - 0 1585243280819 2 connected 5461-10922

5.3查看集群状态
[root@dev1 ~]# /opt/redis5-cluster/bin/redis-cli -c -h 127.0.0.1 -p 7001 -a redis508 cluster info
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_ping_sent:1978
cluster_stats_messages_pong_sent:1994
cluster_stats_messages_sent:3972
cluster_stats_messages_ping_received:1989
cluster_stats_messages_pong_received:1978
cluster_stats_messages_meet_received:5
cluster_stats_messages_received:3972

5.3停止redis集群

这个可以写到脚本里，（-c可以不写）
[root@dev1 ~]# /opt/redis5-cluster/bin/redis-cli -c -h 127.0.0.1 -p 7001 -a redis508 shutdown
[root@dev1 ~]# /opt/redis5-cluster/bin/redis-cli -c -h 127.0.0.1 -p 7002 -a redis508 shutdown
[root@dev1 ~]# /opt/redis5-cluster/bin/redis-cli -c -h 127.0.0.1 -p 7003 -a redis508 shutdown
[root@dev1 ~]# /opt/redis5-cluster/bin/redis-cli -c -h 127.0.0.1 -p 7004 -a redis508 shutdown
[root@dev1 ~]# /opt/redis5-cluster/bin/redis-cli -c -h 127.0.0.1 -p 7005 -a redis508 shutdown
[root@dev1 ~]# /opt/redis5-cluster/bin/redis-cli -c -h 127.0.0.1 -p 7006 -a redis508 shutdown

5.4cluster帮助

127.0.0.1:7003> cluster help
1) CLUSTER <subcommand> arg arg ... arg. Subcommands are:
2) ADDSLOTS <slot> [slot ...] -- Assign slots to current node.
3) BUMPEPOCH -- Advance the cluster config epoch.
4) COUNT-failure-reports <node-id> -- Return number of failure reports for <node-id>.
5) COUNTKEYSINSLOT <slot> - Return the number of keys in <slot>.
6) DELSLOTS <slot> [slot ...] -- Delete slots information from current node.
7) FAILOVER [force|takeover] -- Promote current replica node to being a master.
8) FORGET <node-id> -- Remove a node from the cluster.
9) GETKEYSINSLOT <slot> <count> -- Return key names stored by current node in a slot.
10) FLUSHSLOTS -- Delete current node own slots information.
11) INFO - Return onformation about the cluster.（这个information写错了？？）
12) KEYSLOT <key> -- Return the hash slot for <key>.
13) MEET <ip> <port> [bus-port] -- Connect nodes into a working cluster.
14) MYID -- Return the node id.
15) NODES -- Return cluster configuration seen by node. Output format:
16) <id> <ip:port> <flags> <master> <pings> <pongs> <epoch> <link> <slot> ... <slot>
17) REPLICATE <node-id> -- Configure current node as replica to <node-id>.
18) RESET [hard|soft] -- Reset current node (default: soft).
19) SET-config-epoch <epoch> - Set config epoch of current node.
20) SETSLOT <slot> (importing|migrating|stable|node <node-id>) -- Set slot state.
21) REPLICAS <node-id> -- Return <node-id> replicas.
22) SLOTS -- Return information about slots range mappings. Each range is made of:
23) start, end, master and replicas IP addresses, ports and ids