Cache routing load balancing of (consistency Hash) algorithm implemented in Java
Distributed system load balancing problem using Hash algorithm when you can let some fixed request falls on the same server, each server so that the fixed handle part of the request (and maintain information on these requests), play a role in load balancing. For example, a distributed cache, since it is cached, there is no need to do that all data on the machine are exactly the same cache cluster, but should design a good buffer routing tools, so consistency Hash algorithm is thus born a.
A measure of consistency Hash algorithm two most important features:
① balance : balance refers to the hash result as possible to be distributed to all of the buffer, so that all of the buffer space may have been utilized.
② Monotonicity : If the monotonicity refers to already have some data assigned to the respective machine through hashing, there are new machines added to the system. Hash result should be able to ensure that the original data either to stay or not move on it in the machine, or be migrated to the new machine, but does not migrate to other machines on the old.
Hash algorithm commonly used two industry consistent Hash algorithm, one is without a virtual node, the other one is the Hash algorithm with virtual node.
Consistency Hash algorithm, the following code is the principle without a virtual node later analysis:
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.util.SortedMap;
import java.util.TreeMap;
import java.util.UUID;
import org.apache.commons.lang.StringUtils;
/**
* 一致性Hash算法
* */
public class ConsistentHashWithoutVirtualNode {
/**
* 服务器节点信息
* */
private static SortedMap<Integer, String> nodeMap = new TreeMap<>();
//服务器配置信息(可配置)
private static String[] servers = {"192.168.56.120:6379",
"192.168.56.121:6379",
"192.168.56.122:6379",
"192.168.56.123:6379",
"192.168.56.124:6379"};
/**
* 初始化
* */
static{
for(int i=0; i< servers.length; i++){
nodeMap.put(getHash(servers[i]), servers[i]);
}
System.out.println("Hash环初始化完成!");
}
/**
* 经典的Time33 hash算法
* */
public static int getHash(String key) {
if(StringUtils.isEmpty(key))
return 0;
try{
MessageDigest digest = MessageDigest.getInstance("MD5");
key = new String(digest.digest(key.getBytes()));
}catch(NoSuchAlgorithmException e){
e.printStackTrace();
}
int hash = 5381;
for (int i = 0; i < key.length(); i++) {
int cc = key.charAt(i);
hash += (hash << 5) + cc;
}
return hash<0 ? -hash : hash;
}
/**
* 缓存路由算法
* */
public static String getServer(String key){
int hash = getHash(key);
//得到大于该Hash值的所有Map
SortedMap<Integer, String> subMap = nodeMap.tailMap(hash);
if(subMap.isEmpty()){
int index = nodeMap.firstKey();
System.out.printf("%s被路由到节点[%s]\n", key, nodeMap.get(index));
return nodeMap.get(index);
}else{
int index = subMap.firstKey();
System.out.printf("%s被路由到节点[%s]\n", key, nodeMap.get(index));
return nodeMap.get(index);
}
}
/**
* 使用UUID模拟随机key
* */
public static void main(String[] args) {
for(int i=0; i<20; i++){
String str = UUID.randomUUID().toString();
getServer(str);
}
}
}
复制代码First, the balance, we need to choose a good Hash function, Hash algorithm we chose the more well-known within the industry Time33 Hash algorithms that you can Baidu. But Time33 Hash algorithm has a drawback, and that is key for almost two strings, they generated Hash value is very close, so our solution is to take a fingerprint information using the MD5 algorithm before generating Hash values.
nodeMap server node is used to store information SortedMap (key is a hash value, value information server node); Servers configuration information server; static code block using the static initialization nodeMap stored node information.
Cache routing algorithm is the core code, probably idea is to calculate the hash key value, then all key nodeMap larger than the hash value of the key with a hash value is found, if found, to take a minimum of key-value pairs that key-value pairs as a result the value of the route; key if the key is not found on the hash key value is greater than the right, then take the minimum value nodeMap key in key-value pairs as a result of the route.
Next, we use UUID generate a random string test it, test results are as follows:
So we choose to a virtual machine is divided into a number of nodes, and the virtual nodes distributed over a cross hash ring, through improved code as follows:
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.util.SortedMap;
import java.util.TreeMap;
import java.util.UUID;
import org.apache.commons.lang.StringUtils;
public class ConsistentHashWithVirtualNode {
/**
* 虚拟节点信息
* key:hash值
* value:真实节点+"&"+序号
* */
private static SortedMap<Integer, String> virtualNodeMap = new TreeMap<>();
//单机虚拟节点
private static final int VIRTUAL_NODE_NUM = 5;
//服务器配置信息(可配置)
private static String[] servers = {"192.168.56.120:6379",
"192.168.56.121:6379",
"192.168.56.122:6379",
"192.168.56.123:6379",
"192.168.56.124:6379"};
/**
* 初始化
* */
static{
for(int i=0; i< servers.length; i++){
for(int j=0; j<VIRTUAL_NODE_NUM; j++){
String virtualNodeName = servers[i] + "&" + j;
virtualNodeMap.put(getHash(virtualNodeName), virtualNodeName);
}
}
System.out.println("带虚拟节点的Hash环初始化完成!");
}
/**
* 经典的Time33 hash算法
* */
public static int getHash(String key) {
if(StringUtils.isEmpty(key))
return 0;
try{
MessageDigest digest = MessageDigest.getInstance("MD5");
key = new String(digest.digest(key.getBytes()));
}catch(NoSuchAlgorithmException e){
e.printStackTrace();
}
int hash = 5381;
for (int i = 0; i < key.length(); i++) {
int cc = key.charAt(i);
hash += (hash << 5) + cc;
}
return hash<0 ? -hash : hash;
}
/**
* 缓存路由算法
* */
public static String getServer(String key){
int hash = getHash(key);
//得到大于该Hash值的所有Map
SortedMap<Integer, String> subMap = virtualNodeMap.tailMap(hash);
if(subMap.isEmpty()){
int index = virtualNodeMap.firstKey();
System.out.printf("%s被路由到虚拟节点[%s]真实节点[%s]\n", key, virtualNodeMap.get(index),
virtualNodeMap.get(index).substring(0, virtualNodeMap.get(index).indexOf("&")));
return virtualNodeMap.get(index).substring(0, virtualNodeMap.get(index).indexOf("&"));
}else{
int index = subMap.firstKey();
System.out.printf("%s被路由到虚拟节点[%s]真实节点[%s]\n", key, virtualNodeMap.get(index),
virtualNodeMap.get(index).substring(0, virtualNodeMap.get(index).indexOf("&")));
return virtualNodeMap.get(index).substring(0, virtualNodeMap.get(index).indexOf("&"));
}
}
/**
* 使用UUID模拟随机key
* */
public static void main(String[] args) {
for(int i=0; i<20; i++){
String str = UUID.randomUUID().toString();
getServer(str);
}
}
}
复制代码Thus, the general idea of using real node virtual node + "&" + number (range virtual node number is the number 0 to the desired VIRTUAL_NODE_NUM single server) as the value of the virtual node, the core code as shown above screenshot .
Next, we use UUID generate a random string test it, test results are as follows:
Okay, you're done, That's all about consistency Hash routing algorithm. PS: the above code is cut , which extracts only the core code. If you like my content, then welcome to forward, thank you.
Welcome to the concern of my micro-channel public number "Java architect to develop mind," from time to time to share all kinds of interview questions, Pit experience.
Reproduced in: https: //juejin.im/post/5cfe6118e51d45772a49ad17