I. Introduction

Redis cluster deployment is mainly used for large-scale cache architectures. Generally, small architectures can use redis master-slave configuration.

Redis cluster can be used to dynamically expand the cluster conveniently and quickly, dynamically add and delete nodes, reshard, and has automatic fault recovery function.

Generally, the redis cluster uses 3 masters and 3 slaves, and try to ensure that the master server and the slave server are not on the same machine to prevent the cluster from being paralyzed due to machine failure. Each master server is equipped with a slave server to ensure the high availability of the cluster.

Official address:

Scaling with Redis Cluster | Redis

2. Cluster planning

Each of the three servers deploys a master node and a slave node, and there is no direct master-slave relationship on the same server.

Redis-cluster
serial number	master slave 1	master slave 2	master slave 3
1	192.168.2.211	192.168.2.212	192.168.2.213
2	192.168.2.214	192.168.2.152	192.168.2.153

Software version:

OS：CentOS7.6
Redis：redis-6.2.6

Compile dependency installation

yum -y install gcc tcl
yum -y install centos-release-scl
yum -y install devtoolset-9-gcc devtoolset-9-gcc-c++ devtoolset-9-binutils

Download, compile and install

cd /opt
wget http://download.redis.io/releases/redis-6.2.6.tar.gz
tar -xvf redis-6.2.6.tar.gz
cd redis-6.2.6
make MALLOC=libc
make install PREFIX=/app/software/redis-6.2.6 -j 4

create folder

mkdir -p /app/software/redis-6.2.6/conf
mkdir -p /app/software/redis-6.2.6/log
mkdir -p /app/software/redis-6.2.6/data/
mkdir -p /app/software/redis-6.2.6/run/
cp redis.conf /app/software/redis-6.2.6/conf/

Open the redis.conf file and modify it to the following:

Other servers also have the same configuration file, just modify the local IP.

#添加本机的ip
bind 192.168.2.211
#端口
port 26379

#守护进程
daemonize yes

#pid存储目录
pidfile /app/software/redis-6.2.6/run/redis_26379.pid
#日志存储目录
logfile /app/software/redis-6.2.6/log/redis_26379.log
#数据存储目录，目录要提前创建好
dir /app/software/redis-6.2.6/data/

#开启集群
cluster-enabled yes
#集群节点的超时时间，单位：ms，超时后集群会认为该节点失败
cluster-node-timeout 15000
#集群节点配置文件，这个文件是不能手动编辑的。确保每一个集群节点的配置文件不同
cluster-config-file nodes_26379.conf

#->@wjw_note: 一下是根据实际情况来填写
# Close the connection after a client is idle for N seconds (0 to disable)
timeout 0
loglevel warning
databases 16
# 设置Redis占用内存的大小
maxmemory 4gb

# 如果内存满了就需要按照如相应算法进行删除过期的/最老的
# volatile-lru (Redis3.0之前,默认的内存淘汰策略): 淘汰所有设置了过期时间的键值中，最久未使用的键值
# volatile-lfu (Redis4.0后新增的内存淘汰策略): 淘汰所有设置了过期时间的键值中，最少使用的键值
# allkeys-lru  淘汰整个键值中最久未使用的键值(包含那些未设置过期时间的key)
# allkeys-lfu (Redis4.0后新增的内存淘汰策略): 淘汰整个键值中最少使用的键值
# volatile-random/allkeys-random 随机淘汰设置了过期时间的任意键值/随机淘汰任意键值
# volatile-ttl 根据Time-To-Live移除即将过期的key
# noeviction   永不过期,而是报错
maxmemory-policy volatile-lru

# Redis并不是真正的LRU/TTL,而是基于采样进行移除的,即如采样10个数据移除其中最老的/即将过期的
maxmemory-samples 10

# AOF持久化
appendonly yes

appendfilename appendonly.aof
# 持久化策略,默认每秒fsync一次,也可以选择always即每次操作都进行持久化,或者no表示不进行持久化而是借助操作系统的同步将缓存区数据写到磁盘
appendfsync everysec

# AOF重写策略(同时满足如下两个策略进行重写)
# 当AOF文件大小占到初始文件大小的多少百分比时进行重写
auto-aof-rewrite-percentage 100
# 触发重写的最小文件大小
auto-aof-rewrite-min-size 1gb

# 为减少磁盘操作,暂缓重写阶段的磁盘同步
no-appendfsync-on-rewrite yes

# 慢查
# 下面的时间单位是微秒,所以1000000就是1秒.注意,负数时间会禁用慢查询日志,而0则会强制记录所有命令.
slowlog-log-slower-than 10000
# 这个长度没有限制.只要有足够的内存就行.你可以通过 SLOWLOG RESET 来释放内存.(注:慢查日志是在内存里)
slowlog-max-len 128

masterauth 123456
requirepass 123456

start up

/app/software/redis-6.2.6/bin/redis-server /app/software/redis-6.2.6/conf/redis.conf

cluster creation

3. Deployment result verification

3.1 Node status

Use the cluster nodes command to view the node status.

3.2 Cluster status

Use the cluster info command to view the cluster status.

3.3 Client Verification

Use the client redis-cli binary to access an instance of a server casually, and perform set and get tests.

Precautions

1. If a master node and all its slave nodes are offline, the redis cluster will stop working. The redis cluster does not guarantee the strong consistency of data. Under certain circumstances, the redis cluster will lose the write commands that have been executed.

2. The use of asynchronous replication (asynchronous replication) is one of the reasons why the redis cluster may lose write commands. Sometimes due to network reasons, if the network is disconnected for too long, the redis cluster will start a new master node and send it to the master before. The data of the node will be lost.

Four, redis-cluster-proxy use

4.1 Preface

Redis Cluster internally uses the Gossip protocol in P2P. Each node can obtain services from other nodes or provide services to other nodes. There is no concept of a center, and all information of the entire cluster can be obtained through one node. Therefore, if the application connects to Redis Cluster, one node address can be configured, and multiple node addresses can also be configured. However, it should be noted that if the cluster performs the operation of the upper and lower nodes, its application also needs to be modified, which will lead to the need to restart the application, which is very unfriendly. Since Redis 6.0, Prxoy has been supported, and Proxy can be used directly to manage each cluster node. This article introduces how to use the official proxy: redis-cluster-proxy

4.2 Description

By using redis-cluster-proxy you can communicate with a group of instances that make up a Redis cluster, just like a single instance. The Redis Cluster agent is multi-threaded and uses a multiplexed communication model, so each thread has its own connection to the cluster which is shared by all clients belonging to that thread itself.

In some special cases (such as MULTI transactions or blocking commands), multiplexing will be disabled; and the client will have its own cluster connection. This way the client does not need a dedicated connection to the Redis cluster just to send simple commands like GET and SET.

The main functions of redis-cluster-proxy :

Routing: Every query is automatically routed to the correct node of the cluster
Multithreading
Supports multiplexed and dedicated connection models
In a multiplexed context, query execution and reply ordering can be ensured
Automatically update the cluster's configuration after an ASK|MOVED error: When such an error occurs in a reply, the broker automatically updates the cluster by fetching the cluster's updated configuration and remapping all slots. All queries will be re-executed after the update is complete, so from the client's point of view, everything works fine (clients will not receive ASK|MOVED errors: they will receive the expected reply directly after the request) updated).
Cross-slot/cross-node queries: Supports many commands involving multiple keys belonging to different slots (or even different cluster nodes). These commands will split the query into multiple queries which will be routed to different slots/nodes. Reply handling for these commands is command-specific. Certain commands, such as MGET, will combine all replies as if they were a single reply. Other commands such as MSET or DEL will aggregate the results of all replies. Since these queries effectively break the atomicity of the command, their usage is optional (disabled by default).
Some commands without a specific node/slot (e.g. DBSIZE) will be delivered to all nodes, and the mapped replies will be map-reduced so that the sum of all values contained in all replies is derived.
Additional PROXY commands that can be used to perform certain proxy-specific operations

4.3 Environmental dependencies

1. centos 7.x, you need to manually install gcc+ 8 to support redis-cluster-proxy

2. For reference, if it is centos 8.0 version, the version of gcc 8 has been pre-installed, you can view it through gcc -v

yum install centos-release-scl -y
yum install devtoolset-8-gcc devtoolset-8-gcc-c++ -y
scl enable devtoolset-8 -- bash

4.4 Download and compile

git clone https://github.com/RedisLabs/redis-cluster-proxy.git

cd redis-cluster-proxy
make && make install PREFIX=/opt/redis-cluster-proxy   -j 4

start up

/app/software/redis-cluster-proxy/bin/redis-cluster-proxy  -c /app/software/redis-cluster-proxy/conf/proxy.conf

Client connection verification

Official website : The project is currently alpha code, which is indented and evaluated by the community for suggestions and contributions. We discourage its use in any production environment.

GitHub - RedisLabs/redis-cluster-proxy: A proxy for Redis clusters.

Current status

This project is currently alpha code that is indented to be evaluated by the community in order to get suggestions and contributions. We discourage its usage in any production environment.

Redis6.x cluster construction (4)

Table of contents