Slicing cluster, also called sharding cluster , refers to starting multiple Redis instances to form a cluster, and then dividing the received data into multiple shares according to certain rules, each of which is saved by one instance. Going back to our previous scenario, if the 25GB data is divided into 5 parts (of course, it can be divided evenly), and 5 instances are used for storage, each instance only needs to save 5GB of data. As shown below:
In fact, sliced clusters are a general mechanism for storing large amounts of data, and this mechanism can have different implementations. Before Redis 3.0 , the official did not provide specific solutions for sliced clusters. Starting from 3.0, an official solution called **Redis Cluster is provided to implement slice clusters. **The corresponding rules for data and instances are stipulated in the Redis Cluster scheme.
Specifically, the Redis Cluster solution uses hash slots (Hash Slots, which I will directly call Slots) to handle the mapping relationship between data and instances. In the Redis Cluster scheme, a slicing cluster has a total of 16384 hash slots . These hash slots are similar to data partitions, and each key-value pair is mapped to a hash slot according to its key.
So, how are these hash slots mapped to specific Redis instances?
When we deploy the Redis Cluster solution, we can use the cluster create command to create a cluster. At this time, Redis will automatically distribute these slots evenly on the cluster instances. For example, if there are N instances in the cluster, then the number of slots on each instance is 16384/N.
Of course, we can also use the cluster meet command to manually establish a connection between instances to form a cluster, and then use the cluster addslots command to specify the number of hash slots on each instance.
In addition, I will give you another reminder. When manually assigning hash slots, you need to allocate all 16384 slots, otherwise the Redis cluster will not work properly.
How does the client locate the data?
When locating key-value pair data, the hash slot it is in can be obtained by calculation, and this calculation can be performed when the client sends a request. However, to further locate the instance, it is also necessary to know which instance the hash slot is distributed on.
After the client establishes a connection with the cluster instance, the instance will report its own hash slot information to the client. Each instance does not know the hash slots of other instances at the beginning of the cluster creation.
**So, why can the client get all the hash slot information when accessing any instance? **This is because the Redis instance will send its own hash slot information to other instances connected to it to complete the diffusion of hash slot allocation information. When the instances are connected to each other, each instance has a mapping relationship of all hash slots.
** After the client receives the hash slot information, it will cache the hash slot information locally. **When the client requests a key-value pair, it will first calculate the hash slot corresponding to the key, and then send a request to the corresponding instance.
However, in a cluster, the correspondence between instances and hash slots is not static. There are two most common changes:
-
In the cluster, when instances are added or deleted, Redis needs to reassign hash slots;
-
For load balancing, Redis needs to redistribute hash slots across all instances.
two errors
When the client sends an operation request of a key-value pair to an instance, if there is no hash slot for the key-value pair mapping on the instance, then the instance will return the following MOVED command response result to the client, This result contains the access address of the new instance.
GET hello:key
(error) MOVED 13320 172.16.19.5:6379
Among them, the MOVED command indicates that the hash slot 13320 where the key-value pair requested by the client is located is actually on the instance 172.16.19.5. By returning the MOVED command, it is equivalent to telling the client the information of the new instance where the hash slot is located. In this way, the client can directly connect to 172.16.19.5 and send operation requests.
GET hello:key
(error) ASK 13320 172.16.19.5:6379
The ASK command in this result indicates that the hash slot 13320 where the key-value pair requested by the client is located is on the instance 172.16.19.5, but this hash slot is being migrated. At this point, the client needs to send an ASKING command to the instance 172.16.19.5 first. The meaning of this command is to let this instance allow the execution of the command sent by the client next. Then, the client sends a GET command to this instance to read the data.
The ASK command has two meanings: first, it indicates that the Slot data is still being migrated; second, the ASK command returns the latest instance address of the data requested by the client to the client. At this time, the client needs to send the ASKING command to instance 3 , and then send the operation command.
Unlike the MOVED command, the ASK command does not update the hash slot allocation information cached by the client. The function of the ASK command is only to allow the client to send a request to the new instance, unlike the MOVED command, which changes the local cache so that all subsequent commands are sent to the new instance.