The previous article has introduced the redis installation under windows
First introduce the concept of cluster
Introduction to Redis cluster
Redis cluster is an assembly that provides data sharing among multiple Redis nodes .
Redis cluster does not support commands that process multiple keys, because this requires moving data between different nodes, which cannot achieve the performance like Redis, and may cause unpredictable errors under high load conditions.
The Redis cluster provides a certain degree of availability through partitioning , and continues to process commands in the actual environment when a node is down or unreachable. The advantages of the Redis cluster:
- Automatically split data to different nodes.
- The command can continue to be processed in the case of failure or unreachability of some nodes of the entire cluster.
Redis cluster data sharding
Redis cluster does not use consistent hash, but introduces the concept of hash slot .
The Redis cluster has 16384 hash slots, and each key is checked by CRC16 and modulo 16384 to determine which slot to place. Each node in the cluster is responsible for a part of the hash slot. For example, if the current cluster has 3 nodes, then :
- Node A contains hash slots 0 to 5500.
- Node B contains hash slots 5501 to 11000.
- Node C contains hash slots 11001 to 16384.
This structure is easy to add or delete nodes. For example, if I want to add a new node D, I need to add some slots from nodes A, B, C to D. If I want to remove node A, I need to add Move the slots to the B and C nodes, and then remove the A node without any slots from the cluster. Since moving the hash slot from one node to another node does not stop the service, no matter whether you add, delete or change The number of hash slots of a node will not cause the cluster to be unavailable.
Master-slave replication model of Redis cluster
In order to make the cluster still usable when some nodes fail or most nodes cannot communicate, the cluster uses a master-slave replication model, and each node will have N-1 replicas.
In our example, a cluster with three nodes A, B, and C, without a replication model, if node B fails, the entire cluster will think that it lacks slots in the range of 5501-11000 and is unavailable.
However, if we add a slave node A1, B1, C1 to each node when the cluster is created (or after a period of time), then the entire cluster will consist of three master nodes and three slave nodes, so that after node B fails, The cluster will elect B1 as the new master node to continue serving, and the entire cluster will not be unavailable because the slot cannot be found
But when B and B1 both fail, the cluster is unavailable.
Redis consistency guarantee
Redis does not guarantee strong data consistency . This means that in practice, the cluster may lose write operations under certain conditions.
The first reason is because the cluster uses asynchronous replication. The write operation process:
- The client writes a command to the master node B.
- The master node B replies the command status to the client.
- The master node replicates the write operation to his slave nodes B1, B2 and B3.
The master node’s copying of the command occurs after the command reply is returned, because if each processing command request needs to wait for the replication operation to complete, then the master node processing command request speed will be greatly reduced-we must focus on performance and consistency Make a trade-off between. Note: Redis cluster may provide synchronous writing methods in the future. Another situation in which commands may be lost in a Redis cluster is that the cluster has a network partition, and a client is isolated from a small number of instances including at least one master node.
For example, suppose the cluster contains six nodes A, B, C, A1, B1, and C1, where A, B, and C are the master nodes, A1, B1, and C1 are the slave nodes of A, B, and C, and there is one client The end Z1 assumes that there is a network partition in the cluster, then the cluster may be divided into two parties. Most of the parties include nodes A, C, A1, B1, and C1, and a small portion includes node B and client Z1.
Z1仍然能够向主节点B中写入, 如果网络分区发生时间较短,那么集群将会继续正常运作,如果分区的时间足够让大部分的一方将B1选举为新的master，那么Z1写入B中得数据便丢失了.
注意， 在网络分裂出现期间， 客户端 Z1 可以向主节点 B 发送写命令的最大时间是有限制的， 这一时间限制称为节点超时时间（node timeout）， 是 Redis 集群的一个重要的配置选项：
下面我们来搭建集群
1. 安装Ruby并配置环境
首先我们来配置环境变量 SSL_CERT_FILE首先在这里下载证书 http://curl.haxx.se/ca/cacert.pem, 然后再环境变量里设置 SSL_CERT_FILE 这个环境变量，并指向 cacert.pem 文件。（我这里在D盘建了一个文件夹rubyinstaller，把文件放在了里面）
环境配置好之后安装Ruby，Windows可以安装RubyInstaller，下载地址：
https://rubyinstaller.org/
下载之后解压 解压完之后会有一个Ruby22文件夹，然后进入文件夹中打开命令行依次输入命令进行配置
ruby -v
gem -v
gem sources
gem install redis
2.搭建集群
要让集群正常运作至少需要三个主节点，不过在刚开始试用集群功能时， 强烈建议使用六个节点： 其中三个为主节点， 而其余三个则是各个主节点的从节点。 这里我们搭建一个伪分布式，使用6个redis实例来模拟
1.)首先我们再D盘redis文件夹下面创建6个文件夹 7000~7005 依次代表使用端口号7000~7005
2.)我们在7000~7005这6个文件夹下面新建 redis.conf 文件
Then write the following content in it, it is recommended to copy and paste one after writing, but remember to change the port number to the port number of the corresponding folder
port 7000 (corresponding folder port number) cluster-enabled yes cluster-config-file nodes.conf cluster-node-timeout 5000 appendonly yes
Then redis-server.exe in the redis folder is copied and pasted to the 7000~7005 folder in turn
As shown
3.)Start 6 redis services in sequence
Open the command line under each folder
Enter the command (don't turn off after starting)
redis-server.exe redis.conf
4.) Create a startup cluster
Since the redis-trib.rb file is needed to create and start the cluster, it is a Ruby program that sends special commands to the instance to complete the creation of a new cluster, check the cluster, or reshare the cluster.
This file is not included in the redis installation file of Windows. We need to download Redis from the official website. The Redis version of the official website is the Linux version. In the source code src folder, copy redis-trib.rb to the Redis installation directory on this machine.
You can also go here to download http://download.csdn.net/download/a447332241/9930662
Finally, enter the directory where the redis-trib.rb file is located and execute:
ruby redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005This command is used here to create a new cluster. The option –replicas 1 means that we want to create a slave node for each master node in the cluster. The other parameters that follow are the address list of this cluster instance, 3 masters and 3 slaves
redis-trib will print out an expected configuration for you to see. If you think there is no problem, you can enter yes, and redis-trib will apply this configuration to the cluster so that each node can start communicating with each other.
In this way, our cluster is set up.
5.)Test
Enter redis-cli.exe -h 127.0.0.1 -p 7001 -c under the redis folder
On behalf of us, connect to the redis service of port 7001
Redis cluster data distribution strategy:
A method called hash slot is used to allocate data. The redis cluster allocates 16384 slots by default. When we set a key, we will use the CRC16 algorithm to get the modulo to get the slot, and then divide the key into To the nodes in the hash slot interval, the specific algorithm is: CRC16(key)% 16384
Note that: there must be three master nodes after that, otherwise it will fail when creating a cluster. The slot intervals assumed by the three nodes are:
Node A covers 0-5460;
Node B covers 5461-10922;
Node C covers 10923-16383
Therefore, in the above figure, according to the hash slot algorithm of redis cluster: CRC16('name')%16384
is allocated to the redis service on port 7002.
At this point, the configuration of Redis Cluster on Windows is complete
Redis Chinese document http://www.redis.cn/topics/cluster-tutorial.html
In the next article, we will use jedis to integrate with Spring