Fragmented cluster for Redis cluster installation

1. Why use sharded clusters

Master-slave and sentry can solve the problem of high availability and high concurrent reading. But there are still two unresolved issues:

• Mass data storage problem
• The problem with high-concurrency writing is that the essence of the sentinel mode is still the master-slave mode. In the master-slave mode, we can add slavenodes to expand the read concurrency capability, but there is no way to expand the write and storage capabilities.

The above problems can be solved by using shard clusters. Features of shard clusters:

• There are multiple in the cluster master, each masterstoring different data.
• Each mastercan have multiple slavenodes.
• masterBy pingmonitoring each other's health status.
• Client requests can access any node in the cluster and will eventually be forwarded to the correct node.

2. Introduction to Sharded Clusters

A shard cluster is Redisa combination of multiple master-slave structures, each master-slave structure has a master instance and multiple slave instances. RedisThe sharded cluster can be expanded horizontally when the amount of data is increasing, and the key value is placed in the specified instance, so as to reduce the system's dependence on a single master node, thereby improving the read and write performance of the service Redis.

3. Hash slots

RedisEach masternode will be mapped to a 0~16383total 16384of slots ( hash slot). keyHash slots are similar to data partitions. Each key-value pair will be mapped to a hash slot according to its value . The specific execution process is divided into two steps:

• According to the key-value pair key, CRC16calculate a 16bitvalue according to the algorithm
• Then use 16bitthe value pair to 16384take the modulus to get 0~16383the modulus within the range. Each modulus represents a hash slot with a corresponding number. Each node is responsible for processing a part of the slots. If there are nodes Redisin the cluster , the range of slots each node is responsible for is as follows :masterABC

masternode	Processing slot
A	0-5460
B	5461-10922
C	10923-16383

When viewing the cluster information, you can see:

Note: **data keyis not bound to nodes, but to slots. **The advantage of this binding is that when the cluster expands to increase nodes or downtime reduces masternodes, Redisit is more convenient to transfer the slot to the surviving node, and the data follows the slot transfer, so that we can find the original data location s position.

RedisThe slot value will keybe calculated according to the valid part of the , in two cases:

• keycontains "{}", and "{}" contains at least 1 character, and the part in "{}" is a valid part
• keyDoes not contain "{}", the whole keyis a valid part

4. Build a shard cluster

4.1. Cluster structure

Sharded clusters require a large number of nodes. Here we build a minimal sharded cluster with 3 masternodes, each of which mastercontains one slavenode. The structure is as follows:

Here we will start 6 redisinstances in the same virtual machine to simulate a fragmented cluster. The information is as follows:

IP	PORT	Role
10.0.4.10	7001	master
10.0.4.10	7002	master
10.0.4.10	7003	master
10.0.4.10	8001	slave
10.0.4.10	8002	slave
10.0.4.10	8003	slave

4.2. Prepare instance and configuration

Delete the previous directories 7001, 7002, and 7003, and recreate the directories 7001, 7002, 7003, 8001, 8002, and 8003:

# 进入/home/redis目录
cd /home/redis
# 删除旧的，避免配置干扰
rm -rf 7001 7002 7003
# 创建目录
mkdir 7001 7002 7003 8001 8002 8003

/home/redisPrepare a new redis.conffile below with the following content:

port 6379
# 开启集群功能
cluster-enabled yes
# 集群的配置文件名称，不需要我们创建，由redis自己维护
cluster-config-file /home/redis/6379/nodes.conf
# 节点心跳失败的超时时间
cluster-node-timeout 5000
# 持久化文件存放目录
dir /home/redis/6379
# 绑定地址
bind 0.0.0.0
# 让redis后台运行
daemonize yes
# 注册的实例ip
replica-announce-ip 10.0.4.10
# 保护模式
protected-mode no
# 数据库数量
databases 16
# 日志
logfile /home/redis/6379/run.log

Copy this file into each directory:

# 进入/home/redis
cd /home/redis
# 执行拷贝
echo 7001 7002 7003 8001 8002 8003 | xargs -t -n 1 cp redis.conf

Modify under each directory redis.conf, modify 6379 to be consistent with the directory:

# 进入/home/redis目录
cd /home/redis
# 修改配置文件
printf '%s\n' 7001 7002 7003 8001 8002 8003 | xargs -I{} -t sed -i 's/6379/{}/g' {}/redis.conf

4.3. Startup

Since the background startup mode has been configured, the service can be started directly:

# 进入/home/redis目录
cd /home/redis
# 一键启动所有服务
printf '%s\n' 7001 7002 7003 8001 8002 8003 | xargs -I{} -t redis-server {}/redis.conf

View status via ps:

ps -ef | grep redis

Discovery services have been started normally:

If you want to close all processes, you can execute the command:

ps -ef | grep redis | awk '{print $2}' | xargs kill

or (recommended this way):

printf '%s\n' 7001 7002 7003 8001 8002 8003 | xargs -I{} -t redis-cli -p {} shutdown

4.4. Create a cluster

Although the service is started, each service is currently independent without any association.

We need to execute commands to create a cluster. Redis5.0It was cumbersome to create a cluster before. After 5.0, cluster management commands are integrated into redis-cli.

1) Redis5.0Before

Redis5.0Previous cluster commands were implemented using redisthe installation package . src/redis-trib.rbBecause redis-trib.rbit is rubywritten in a language, an installation rubyenvironment is required.

# 安装依赖
yum -y install zlib ruby rubygems
gem install redis

Then use the command to manage the cluster:

# 进入redis的src目录
cd /home/redis/redis-6.2.4/src
# 创建集群
./redis-trib.rb create --replicas 1 10.0.4.10:7001 10.0.4.10:7002 10.0.4.10:7003 10.0.4.10:8001 10.0.4.10:8002 10.0.4.10:8003

2) Redis5.0Later

We are using Redis6.2.4the version, cluster management and integrated into it redis-cli, the format is as follows:

redis-cli --cluster create --cluster-replicas 1 10.0.4.10:7001 10.0.4.10:7002 10.0.4.10:7003 10.0.4.10:8001 10.0.4.10:8002 10.0.4.10:8003

Command description:

• redis-cli --clusterOr ./redis-trib.rb: represents a cluster operation command
• create: represents the creation of a cluster
• --replicas 1Or --cluster-replicas 1: specify that masterthe number of copies of each in the cluster is 1, and the number 节点总数 ÷ (replicas + 1)obtained at this time is masterthe number. Therefore, the first one in the node list nis masterthat the other nodes are slaveall nodes, randomly assigned to differentmaster

What it looks like after running:

The question here is to create a cluster in the way listed above? If there are no problems, enter yesand the cluster will start to create:

You can view the cluster status with the command:

redis-cli -p 7001 cluster nodes

4.5. Testing

Try to connect to node 7001 and store a piece of data:

# 连接
redis-cli -p 7001
# 存储数据
set num 123
# 读取数据
get num
# 再次存储
set a 1

The result is tragic:

During cluster operation, you need to redis-cliadd -cparameters:

redis-cli -c -p 7001

This time it works: