Article directory
Consistent Hash Principle
In simple terms, the consistent Hash algorithm will
整个哈希值空间组织成一个虚拟的圆环,
for example, assume that the value space of a certain hash function H is0 ~ 2^32-1(that is, the hash value is a 32-bit unsigned integer)
Improve fault tolerance and scalability
1. When the user visits, use the same function Hash to calculate the hash value according to the user's IP, and determine the position of this data on the ring. From this position, walk clockwise along the ring, and the first server encountered is the server that should be The located server.
2. Pay attention to improving fault tolerance here.用到了虚拟节点设置在圆环上The essence of this virtual node is a node that is bound to a physical node. What comes to mind here is the binding in the form of a map
Consistent Hash Implementation
train of thought
1. The hash value is a non-negative integer, and the value range of the non-negative integer is made into a ring;
2. Find the hash value (such as the node name) for a certain attribute of the node in the cluster, and put the node into the ring according to the hash value 3.
Find the hash value for the key of the data, and put the data on the ring in the same way按顺时针方向，找离他最近的节点，就存储到这个节点上. (This ring is equivalent to an ordered collection, and for the convenience of searching, the storage structure uses treeMap)
The data node structure stored in the ring is shown in the figure below:
the code
public class Node {
private String id;
private String name;
private List<String> datas = new ArrayList<>();
public Node() {
}
public Node(String id, String name) {
this.id = id;
this.name = name;
}
public void setId(String id) {
this.id = id;
}
public void setName(String name) {
this.name = name;
}
public String getId() {
return id;
}
public String getName() {
return name;
}
public void setDatas(List<String> datas) {
this.datas = datas;
}
public List<String> getDatas() {
return datas;
}
@Override
public String toString() {
return "Node【" +
"id='" + id + '\'' +
", name='" + name + '\'' +
", datas=" + datas.toString() +
'】';
}
}
@Data
public class HashUtilServe {
// 存放物理节点
private List<Node> nodeList = new ArrayList<>();
// 设置每个物理节点的虚拟节点50个
private int virtalNodeNum = 50;
// 物理节点和虚拟hash节点key进行关联
private HashMap<Node,List<Integer>> virNode = new HashMap<>();
// 定义一个hash集合存放节点
private SortedMap<Integer,Node> sortedMap = new TreeMap<>();
/**
* 增加服务节点
* @param node
*/
public void createServe(Node node){
// 加入物理节点
nodeList.add(node);
ArrayList<Integer> hashlist = new ArrayList<>();
// 创建虚拟节点
for(int i=0; i<virtalNodeNum; i++){
int hashValue = FNVHash1(node.getId() + "-" + i);
hashlist.add(hashValue);
sortedMap.put(hashValue,node);
}virNode.put(node,hashlist);
}
/**
* 删除服务节点(模拟物理节点宕机)
* @param node
*/
public void deleteServe(Node node){
// 删除物理节点
nodeList.remove(node);
// 删除对应的物理节点
List<Integer> hashs = virNode.get(node);
for (Integer hash : hashs) {
sortedMap.remove(hash);
}
// 删除关联表
virNode.remove(node);
}
/**
* 查好数据对应服务节点
* @param data
* @return
*/
public Node findServe(String data){
// 对数据进行hash
int hashValue = FNVHash1(data);
// 获取【key>=hashvalue】的虚拟节点的map
SortedMap<Integer, Node> findedMap = sortedMap.tailMap(hashValue);
if(findedMap.isEmpty()){
// 只有一台服务器节点并且当前数据缓存的服务器宕机了，找不到服务节点
return null;
// return sortedMap.get(sortedMap.firstKey());
}
// 拿最近一台的服务器节点
return findedMap.get(findedMap.firstKey());
}
/**
* 分布式缓存存储数据
* @param data
* @return
*/
public Node saveData(String data){
// 对数据进行hash
int hashValue = FNVHash1(data);
// 获取【key>=hashvalue】的虚拟节点的map
SortedMap<Integer, Node> findedMap = sortedMap.tailMap(hashValue);
Node node =new Node();
if(findedMap.isEmpty()){
// 只有一台服务器节点
node = sortedMap.get(sortedMap.firstKey());
}
// 拿最近一台的服务器节点
node = findedMap.get(findedMap.firstKey());
// 将数据缓存到对应服务节点中
List<String> datas = node.getDatas();
datas.add(data);
node.setDatas(datas);
return node;
}
// 散列工具类
public int FNVHash1(String data) {
final int p = 16777619;
int hash = (int) 2166136261L;
for (int i = 0; i < data.length(); i++)
hash = (hash ^ data.charAt(i)) * p;
hash += hash << 13;
hash ^= hash >> 7;
hash += hash << 3;
hash ^= hash >> 17;
hash += hash << 5;
//如果算出来的值为负数，则取其绝对值
if(hash < 0){
hash = Math.abs(hash);
}
return hash;
}
}
public class DemoPract {
public static void main(String[] args) {
// 创建三台服节点对象
Node serv1 = new Node("192.168.0.1", "serv1");
Node serv2 = new Node("192.168.0.5", "serv2");
Node serv3 = new Node("192.168.0.11", "serv3");
HashUtilServe hashUtil = new HashUtilServe();
// 将服务器添加到Hash环上
hashUtil.createServe(serv1);
hashUtil.createServe(serv2);
hashUtil.createServe(serv3);
// 缓存数据到服务节点上
hashUtil.saveData("1今天是个好日子");
hashUtil.saveData("2理想要有的");
hashUtil.saveData("3很厉害的事情");
hashUtil.saveData("啊哈java才是最牛的");
Node serve33 = hashUtil.findServe("3很厉害的事情");
System.out.println(serve33.toString());
// 删除服务节点3
hashUtil.deleteServe(serv3);
System.out.println("---------删除服务3节点----------------");
Node serve1 = hashUtil.findServe("1今天是个好日子");
Node serve2 = hashUtil.findServe("2理想要有的");
Node serve3 = hashUtil.findServe("3很厉害的事情");
System.out.println(serve1.toString());
System.out.println(serve2.toString());
System.out.println(serve3.toString());
}
}
The test results are as follows:
As shown in the figure: You can see that after server 3 is down, you can query the data originally cached to service 3, and find that the cached data of server 3 cannot be queried
