Redis as a high performance in-memory database, supporting micro-Bo one hundred million social networking platform, has also become standard on many Internet companies. Here will Redis Cluster cluster core, based on the latest version of Redis5, and then from the principle of combat, Fun Redis cluster
Common Redis clustering solutions
Before introducing Redis Cluster clustering solutions, in order to facilitate comparison, the industry's briefly look at common Redis cluster program:
1 client-based fragmentation
Redis Sharding out before Redis Cluster, multi Redis instances in the cluster method commonly used in the industry. The main idea is based on the hash algorithm, based on a hash value of key data Redis data slice, the data is mapped to the respective node
Advantage is simple, the disadvantage is that when Redis cluster adjustment, each client needs to be updated to adjust
2 fragment based on a proxy server
The client sends a request to the independent deployment of a proxy component, the proxy client component parses the data, and forwards the request to the correct node, and then return the results to the client
Advantage is that transparent access, easy to extend the cluster, the disadvantage that the multi-layer forwarding agent, has performance loss
3 Redis Sentinel (Sentinel)
Redis Sentinel is a high-availability solutions from Redis 2.6 official version provided in Redis master-slave replication cluster on the basis of increased monitoring of the entire cluster Sentinel Redis cluster. When Redis cluster master node fails, Sentinel failover, elect a new master, while Sentinel itself supports high availability cluster deployment
Advantage is that support high availability cluster, high reading and writing, there is no disadvantage that achieve fragmentation of data, each node needs to carry a complete data set, when a load capacity by limiting Redis server, supports only achieved by increasing the vertical expansion of the machine's memory, does not support horizontal expansion
Redis Cluster Design
1 overall design
Redis Cluster version 3.0 is officially launched in the high availability cluster solutions compared Redis Sentinel, Redis Cluster scheme does not require the deployment of additional Sentinel cluster, but the cluster achieved by monitoring internal communications cluster, switching from the main fault; at the same time, based on internal support hash for data fragmentation, the level of support for dynamic expansion
The overall structure is as follows:
A plurality of master nodes in the cluster, each master node from the plurality of nodes, the main coincidence of data between nodes, a minimum of three master nodes, each node requires a minimum of a master node from
- Availability : When the master node fails, automatically switches from the main
- High performance : the master node to provide service to read and write, a read-only from the service node to improve system throughput
- Scalability : data pieces stored in the cluster, various data between the master node, the data corresponding to each maintenance, expansion can be added to the cluster nodes, some nodes may be offline horizontally volume reduction
2 data pieces
The entire set of data according to certain rules assigned to the plurality of nodes, referred to as a data piece, slicing scheme uses a hash Redis Cluster slice
The basic principle is as follows:
the Redis the Cluster numbers are first defined interval from 0 to 16383, referred to as the groove, all the keys are mapped to an integer of from 0 to 16383 slot hash function, the calculation formula: slot = CRC16 (key) & 16383. Each node is responsible for maintaining the mapping part of the groove and the groove of key data
Redis slot is the basic unit of data management cluster , the cluster expansion and contraction is the mobile data channel between the nodes
Mapping relationship between the groove and the node follows:
- Each cluster node maintains a 16384 bit (2kB) group of bits, each bit corresponds to the same number of slots, with 0/1 identifies whether it has a slot for yourself
- Cluster node also maintains a mapping of slots to nodes of the cluster, the length is 16384, an array of array subscripts represent slot numbers, the value of the node information
3 cluster expansion
Redis Cluster support case does not affect the external services of the cluster, the cluster dynamic capacity expansion or contraction, when Redis new node joins an existing cluster, the need for data migration trough and, after migration to ensure that each node is responsible for a similar number of slots the distribution of data on each node
The entire data migration involves a series of operations, Redis provides cluster management tools, including Ruby-based redis-trib.rb, also Redis5 new offer based on the C language redis-cli, the following describes an example to redis-cli
The migration source node to a target node specified data slot, the basic process is as follows:
- (1) redis-cli set the target node designated slot state importing, so that the target node is ready to move into slot data
- (2) redis-cli source node setting state specified slot migrating, so that the source node is ready to move out so that data slot
- (3) redis-cli specified bulk data migration source node to a target node in slot
- (4) After the data migration redis-cli is assigned to a cluster target node to the master node notifies all the grooves, the master node and the node updates the mapping relationship information slot
Typically, if the data is not requested by the client node, the node will respond MOVED redirection information, then the client requests information based on the correct node. For the slot of data being migrated, ensure that the client can still access the normal design as follows:
- (1) is not updated until the migration is complete with slot node mapping relationship information, if the same mapping information before migrating be maintained and migration
- (2) if the client node access source, the access key has not been moved out of the normal processing of the key
- (3) If the client access to the source node, key access yet been moved out, the source node to return redirection information ASK
- (4) The client redirection ASK abnormality information extracted target node, the target node sends ASKING Xianxiang request operation command, then the command execution key
MOVED these two ASK and redirect the following differences:
- ASK redirect Description Cluster slot ongoing data migration, the client can not know when the migration is complete, it can only be temporary redirect, the client does not update the map cache slot to Redis nodes.
- MOVED redirection instructions corresponding to the key slot has been clearly assigned to the new node, it is necessary to update the slot map cache node Redis
4 CAP-offs
CAP include: Consistency (Consistency), availability (Availability), partition fault tolerance (Partition tolerance), the system can not be reached if the data consistency within the time limit, it means that the situation partition took place, must be done between C and A the selection
Redis Cluster chose CA architecture, in order to ensure availability, Redis does not guarantee strong consistency, there will be data inconsistencies under certain conditions even lost a write operation
The first reason is: In order to make a trade-off between performance and consistency, master-slave synchronization of data between nodes is asynchronous replication, when a client successfully written to the master node, master returns success, master node before the write operation to the asynchronous replication slave node
Another reason is that when sending network partition cluster, the cluster may be divided into two parts: the majority and the minority, if masterA node located in the minority, if a network partition occurs a short time, then the cluster will continue to operate normally; if the partition the time is long enough so that the majority in the election for the new alternative matsterA master, then write masterA during partition data is lost
During network partition, the client can send the write command to matsterA maximum time is limited, this limit is called timeout node (cluster-node-timeout), is an important cluster configuration options Redis
to sum up
So far, Redis Cluster clustering concept introduced here, there is also a cluster of communication protocols, memory, data backup, it is worth learning from the master copy and other features, is a model of the design of distributed systems, have the opportunity to expand introduced
Next, introduce Redis Cluster cluster combat
reference
Microblogging six years redis practice
https://mp.weixin.qq.com/s/dBWIHwfmrs6Tt7INw-zSyA
Redis official website describes cluster design specifications
https://redis.io/topics/cluster-tutorial
https://redis.io/topics/cluster-spec
