Redis High Availability Cluster Construction and Data Fragmentation

1. Introduction

1. Redis cluster:

Redis Cluster is a way to spread data and load across multiple nodes. Each node is an independent Redis instance that works together to provide high reliability and performance. In the Redis cluster, data is divided into slots, and these slots are allocated to different node storage, and data migration will be performed automatically when reading and writing data. This method of Redis cluster can achieve linear expansion without using resources such as shared storage or shared memory. In addition, Redis cluster also supports failure recovery and single point of failure automatic transfer.

2. Cluster construction:

Redis clusters are especially necessary when a single Redis instance cannot meet business needs. Redis clusters can improve system availability, throughput, and fault tolerance to meet high-concurrency and high-reliability business scenarios.

2. Redis cluster construction

1. Stand-alone Redis upgrade to Redis Cluster

a. Construction method

Standalone Redis can be upgraded to Redis Cluster to implement cluster deployment. After the upgrade is complete, Redis will become a multi-node cluster, which contains multiple nodes, and each node is a separate Redis instance. Here's how to upgrade Redis from stand-alone mode to Redis Cluster:

1. Install Redis Server and Redis CLI

2. Modify the Redis configuration file redis.conf.

 # 启用集群模式
  cluster-enabled yes
  
  # 节点间通信的超时时间
  cluster-node-timeout 5000
  
  # 集群状态持久化方式，可以是rdb或者aof，也可以都开启
  cluster-config-file nodes-6379.conf

3. Restart the Redis server

4. Create a Redis Cluster using redis-cli

First start a node:

redis-server --port 7001 --cluster-enabled yes --cluster-config-file nodes.conf --cluster-node-timeout 5000

Then use the cluster command of the redis-cli tool on this node to create a cluster:

redis-cli --cluster create 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 …

b. Construction method description

The above construction methods are suitable for small Redis cluster construction. For large-scale production environments, Redis cluster management software (such as Redis Sentinel or Redis Cluster) is required for failover and load balancing.

2. Environment preparation

Before building a Redis cluster, the following preparations are required:

1. Install Redis server and Redis cli tools

2. Install Redis on multiple machines and make sure they are all able to communicate with each other.

3. Make sure cluster-enabled=yes in the configuration file of each Redis Server instance.

4. Create a directory for storing Redis Cluster node data (such as /opt/redis/cluster/n1, /opt/redis/cluster/n2, etc.).

3. Configuration modification

1. Modify the configuration file of each Redis Server instance, and set a series of node parameters (nodeport, nodename, dbfilename, etc.).

   
     # 集群名称
   cluster-name mycluster
   
   # 集群节点列表（多个Redis实例的 IP：PORT 组合）
   cluster-announce-ip <your_ip_address>
   cluster-announce-port <cluster_start_port>
   
   # 同一节点的多个 Redis 实例需要有不同的端口号
   port <different_port>
   
   # 集群工作目录，用于存储会产生的日志和状态信息等
   dir /path/to/redis/node1
   
   # 不使用AOF持久化方式（至少在这里不启动），可以降低内存使用率
   appendonly no
   

2. Copy the configuration files of each Redis Server instance to the corresponding directory (such as /opt/redis/cluster/n1, /opt/redis/cluster/n2, etc.), and multiple directories represent different nodes.

3. Modify the minimum and maximum number of nodes in the cluster (thus determining the number of slots that can be stored).

   cluster-node-timeout 5000 # 节点超时时间（单位毫秒）
   cluster-config-file nodes-6379.conf # 集群状态持久化文件
   cluster-minimum-nodes <number_of_nodes> # 集群最少节点数
   cluster-max-redirects 6 # 节点内部重定向的最大次数
   

4. Start the cluster

1. Start Redis Server instances on multiple nodes and enable "cluster mode" in their configuration files.

2. Use the cluster command of the redis-cli tool to create a cluster:

   redis-cli --cluster create <ip1>:<port1> <ip2>:<port2> ... --cluster-replicas
   

Here <ip>and <port>are the IP addresses and port numbers of each Redis Server instance, and --cluster-replicasthe parameters indicate the use of automatically generated slave nodes.

So far, a Redis cluster has been successfully built. You can use the redis-cli tool for verification and testing.

3. Realization of Redis data fragmentation

Redis data sharding concept description

Redis is a high-performance key-value pair (key-value) storage database that supports small memory. With the gradual increase of business data, stand-alone Redis may have insufficient memory and performance bottlenecks. Data sharding is a common solution to Redis single point of failure, which divides data into multiple nodes for storage and processing.

Data Fragmentation Strategy

Redis officially provides three strategies for data sharding: hash consistency algorithm, sharding by range interval, and hash slot mode sharding.

hash consensus algorithm

The hash consistency algorithm is an algorithm that performs hash operations on data nodes and maps the results to the ring space. In layman's terms, each node is represented by a hash value, and then the value closest to the node is found within a certain range, and the data is stored on the node. It should be noted that when the number of nodes changes, the algorithm needs to recalculate the hash value.

Shard by Range

Fragmentation by range interval is a method of dividing data into several segments according to the value range, and then assigning each segment to a different node for storage and processing. This strategy is suitable for scenarios where the data can be divided into multiple intervals in a certain way. For example, sharding by range of user IDs.

Hash slot mode sharding

Hash slot mode sharding is the most commonly used sharding strategy in Redis clusters. Fix the number of Hash Slots to 16384, use the same hash function for all keys in the database to perform hash operations, and then take the remainder of 16384 to determine the hash slot corresponding to the Key. Each node is responsible for processing a subset of hash slots. If the number of nodes changes, the mapping relationship between slots and nodes will be recalculated, and the data will be migrated to the new nodes.

4. Operation and maintenance of Redis cluster

Monitoring of cluster node status

Redis provides the tools redis-cli, redis-trib.rb, and redis-stat for monitoring cluster status. You can view the real-time status, data flow, QPS and other information of the cluster through the command line or web interface.

Cluster failure handling

When a node in the cluster fails, it can be handled through the following steps: first, check the operation log to analyze the cause of the failure; second, try to restart and fix the problem; finally, if it cannot be repaired, replace the original node by adding a new node.

Node offline recovery

When a node goes offline, it is necessary to confirm whether the node is really offline instead of a false failure, then check the operation log, and perform data migration if necessary.

Slot Migration Handling

When adding or deleting nodes, some slots need to be reallocated to other nodes and data migration is performed. This operation needs to be performed when the cluster is in a normal state. The key is to avoid data duplication and data loss.

Cluster expansion and contraction

The expansion of the cluster is very simple, just need to add new nodes, and then re-allocate the slots and migrate data. Scaling down is relatively difficult, and it is necessary to consider how to perform remote backup, data archiving and other operations. You can use the persistent method to archive the data of the reduced nodes, and restore them through the Restore command provided by Redis.

Five, Redis cluster to achieve high availability solution

1. Redis master-slave architecture

In the Redis master-slave architecture, one node (Redis instance) is designated as the master node (master), and the remaining nodes are slave nodes (slave).
The master node is responsible for the write operation and synchronizes the write operation to the slave node, and the slave node can only read data.

2. Master-slave synchronization

Master-slave synchronization means that the master node synchronizes write operations to the slave nodes to achieve consistency between the slave node data and the master node data. When a slave node loses contact with the master node, the slave node becomes unavailable.

3. Master-slave switching

Master-slave switching refers to switching the master node to a slave node after a failure to ensure system availability.

4. Redis sentinel mode

The Redis sentinel mode is a way to ensure the high availability of Redis. The status of the Redis node of the system is monitored through Sentinel (sentinel). When a Redis node fails, Sentinel will automatically perform master-slave switching.

5. Redis cluster integration practice case

The Redis cluster is to solve the problem of limited capacity of a single machine. Each node in the cluster (which can be regarded as a "shard" in a certain sense) processes part of the data, and all nodes form a complete Redis service.
Below we demonstrate through an example.

Cluster construction

1. Install Redis and Ruby environment:

   sudo apt-get update
   sudo apt-get install redis-server ruby
   

2. Install ruby ​​package manager:

   sudo apt-get install rubygems
   

3. Install redis and redis-trib.rb

   sudo gem install redis
   wget http://download.redis.io/redis-stable/src/redis-trib.rb
   chmod +x redis-trib.rb
   

4. Create 6 Redis instances and start them: redis-6379, redis-6380, redis-6381, redis-6382, redis-6383, redis-6384

5. Create a cluster:

   ./redis-trib.rb create --replicas 1 \
               127.0.0.1:6379 \
               127.0.0.1:6380 \
               127.0.0.1:6381 \
               127.0.0.1:6382 \
               127.0.0.1:6383 \
               127.0.0.1:6384
   

cluster use

1. Connect to any node in the Redis cluster:

   redis-cli -c -h 127.0.0.1 -p 6379
   

2. Create a key in the Redis cluster and check the distribution:

   set foo bar
   cluster keyslot foo
   

3. View the current cluster node status:

   cluster nodes