table of Contents
1, redis cluster Introduction
2, the most old-fashioned and drawbacks of hash algorithm (a lot of caching reconstruction)
3, consistent hash algorithm (automatic cache migration) + virtual node (automatic load balancing)
Do not traverse - "hash algorithm: cache location = hash (key)% n
Add / reduce node - "Cache failure location -" the hash ring
Ring node hash less - "Data inclined -" added virtual node
4, redis cluster algorithm of hash slot
Addressing Distributed Algorithms
- hash algorithm (a lot of caching reconstruction)
- Consistency hash algorithm (automatic cache migration) + virtual node (automatic load balancing)
- redis cluster algorithm of hash slot
1, redis cluster introduce
redis cluster
(1) Automatic data fragmentation, the discharge portion of the data on each master
(2) to provide built-in support for high availability, the partial master is unavailable, or continue to work
In redis cluster architecture, each redis to open two port numbers, for example, is a 6379, is to add another port number 10000, for example 16379
16379 is the port number used for inter-node communication, which is the cluster bus thing, cluster bus. cluster bus communication, used for fault detection, configuration update authorization failover
another cluster bus with the binary protocol used for efficient exchange of data between the nodes, use less network bandwidth and processing time
2, the most old-fashioned hash algorithm (drawbacks: a large cache rebuild)
It is a key, a hash value is first calculated, and then modulo the number of nodes. Then play on a different master node. Once a certain master node goes down, all requests over, are based on the latest master nodes to get a remaining mold, attempts to fetch data. This causes most of the requests come, not to get all valid cache, resulting in a lot of traffic influx database.
3, consistent hash algorithm (automatic cache migration) + virtual node (automatic load balancing)
Make a circle, to resolve the problem shooting.
Consistency hash algorithm hash value of the entire space into a virtual ring, the entire space organization in a clockwise direction, the next step will each master node (using the server's ip or host name) hash. This will determine the location of each node on the ring thereof hash.
To a key, the hash value is first calculated, and determines the position of the data on the ring, from the position of "walking" the ring in a clockwise direction, the first node encountered is a master key location.
In consistent hashing algorithms, if a node linked to the affected data is only the data between this node to one node (the first node encountered in traveling in the counterclockwise direction) before the ring space, no other Affected. Add a node also empathy.
However, consistent hashing algorithm when the node is too easy because the node cache problems caused by uneven distribution of hot spots. In order to solve this hot issue, consistent hash algorithm introduces virtual node mechanism that calculates a plurality of hash for each node, each of the calculation results are placed in the position of a virtual node. This achieves a uniform distribution of data and load balancing. Consistency hash algorithm is a more detailed look at this: consistency Hash Algorithm
4, redis cluster algorithm of hash slot
redis cluster have fixed 16384 hash slot, calculates CRC16 value for each key, and then modulo 16384, may obtain a corresponding key hash slot. Each node is responsible for maintaining a portion of the slot and key data channel is mapped.
redis cluster 中每个 master 都会持有部分 slot，比如有 3 个 master，那么可能每个 master 持有 5000 多个 hash slot。hash slot 让 node 的增加和移除很简单，增加一个 master，就将其他 master 的 hash slot 移动部分过去，减少一个 master，就将它的 hash slot 移动到其他 master 上去。移动 hash slot 的成本是非常低的。客户端的 api，可以对指定的数据，让他们走同一个 hash slot，通过 hash tag 来实现。
举个例子
如上图
当前集群有 5 个节点，每个节点平均大约负责 3276 个槽。由于采用高质量的哈希算法，每个槽所映射的数据通常比较均匀，将数据平均划分到 5 个节点进行数据分区。Redis Cluster 就是采用虚拟槽分区。
节点1： 包含 0 到 3276 号哈希槽。
节点2：包含 3277 到 6553 号哈希槽。
节点3：包含 6554 到 9830 号哈希槽。
节点4：包含 9831 到 13107 号哈希槽。
节点5：包含 13108 到 16383 号哈希槽。
所以hash slot的好处是可以像磁盘分区一样自由分配槽位，在配置文件里可以指定，也可以让redis自己选择分配，结果均匀。
这种结构很容易添加或者删除节点。如果增加一个节点 6，就需要从节点 1 ~ 5 获得部分槽分配到节点 6 上。如果想移除节点 1，需要将节点 1 中的槽移到节点 2 ~ 5 上，然后将没有任何槽的节点 1 从集群中移除即可。
由于从一个节点将哈希槽移动到另一个节点并不会停止服务，所以无论添加删除或者改变某个节点的哈希槽的数量都不会造成集群不可用的状态.
缓存的key hash结果是和slot绑定的，而不是和服务器节点绑定，所以节点的更替只需要迁移slot即可平滑过渡。
出处链接：
https://www.jianshu.com/p/fe7b7800473e
https://www.jianshu.com/p/90b3de6288c6