Article directory
1. Overview of load balancing algorithm
Load balancing, the English name is Load Balance, and its meaning refers to 将负载（工作任务）进行平衡、分摊到多个操作单元上进行运行, for example, FTP servers, Web servers, enterprise core application servers and other main task servers, etc., so as to complete work tasks in coordination. Since multiple machines are involved, it is involved 任务如何分发. This is the problem of load balancing algorithm.
2. RoundRobin algorithm
1 Overview
Polling is queuing up, one after the other.
2. Java implements polling algorithm
Manually write a doubly linked list to implement the request polling algorithm for the server list.
public class RR {
//当前服务节点
Server current;
//初始化轮询类，多个服务器ip用逗号隔开
public RR(String serverName){
System.out.println("init server list : "+serverName);
String[] names = serverName.split(",");
for (int i = 0; i < names.length; i++) {
Server server = new Server(names[i]);
if (current == null){
//如果当前服务器为空，说明是第一台机器，current就指向新创建的server
this.current = server;
//同时，server的前后均指向自己。
current.prev = current;
current.next = current;
}else {
//否则说明已经有机器了，按新加处理。
addServer(names[i]);
}
}
}
//添加机器
void addServer(String serverName){
System.out.println("add server : "+serverName);
Server server = new Server(serverName);
Server next = this.current.next;
//在当前节点后插入新节点
this.current.next = server;
server.prev = this.current;
//修改下一节点的prev指针
server.next = next;
next.prev=server;
}
//将当前服务器移除，同时修改前后节点的指针，让其直接关联
//移除的current会被回收期回收掉
void remove(){
System.out.println("remove current = "+current.name);
this.current.prev.next = this.current.next;
this.current.next.prev = this.current.prev;
this.current = current.next;
}
//请求。由当前节点处理即可
//注意：处理完成后，current指针后移
void request(){
System.out.println(this.current.name);
this.current = current.next;
}
public static void main(String[] args) throws InterruptedException {
//初始化两台机器
RR rr = new RR("192.168.0.1,192.168.0.2");
//启动一个额外线程，模拟不停的请求
new Thread(new Runnable() {
@Override
public void run() {
while (true) {
try {
Thread.sleep(500);
} catch (InterruptedException e) {
e.printStackTrace();
}
rr.request();
}
}
}).start();
//3s后，3号机器加入清单
Thread.currentThread().sleep(3000);
rr.addServer("192.168.0.3");
//3s后，当前服务节点被移除
Thread.currentThread().sleep(3000);
rr.remove();
}
class Server{
Server prev; // 前驱
Server next; // 后继
String name; // 名称
public Server(String name){
this.name = name;
}
}
}
After initialization, only 1, 2, both polling
3 after joining, 1, 2, 3, three polling
After removing 2, only 1 and 3 polling remain
3. Advantages and disadvantages
The implementation is simple, the machine list can be added and subtracted freely, and the time complexity is O(1).
It is impossible to make biased customization for nodes, and the processing capabilities of nodes cannot be treated differently.
3. Random algorithm
1 Overview
Take one from the list of servables 随机to provide a response.
2. Java implements random algorithm
In the random access scenario, it is suitable to use arrays to achieve random subscript reading more efficiently.
Define an array, and take a random number within the length of the array as its subscript. very simple.
public class Rand {
// 所有服务ip
ArrayList<String> ips ;
//初始化随机类，多个服务器ip用逗号隔开
public Rand(String nodeNames){
System.out.println("init list : "+nodeNames);
String[] nodes = nodeNames.split(",");
//初始化服务器列表，长度取机器数
ips = new ArrayList<>(nodes.length);
for (String node : nodes) {
ips.add(node);
}
}
//请求
void request(){
//下标，随机数，注意因子
int i = new Random().nextInt(ips.size());
System.out.println(ips.get(i));
}
//添加节点，注意，添加节点会造成内部数组扩容
//可以根据实际情况初始化时预留一定空间
void addnode(String nodeName){
System.out.println("add node : "+nodeName);
ips.add(nodeName);
}
//移除
void remove(String nodeName){
System.out.println("remove node : "+nodeName);
ips.remove(nodeName);
}
public static void main(String[] args) throws InterruptedException {
Rand rd = new Rand("192.168.0.1,192.168.0.2");
//启动一个额外线程，模拟不停的请求
new Thread(new Runnable() {
@Override
public void run() {
while (true) {
try {
Thread.sleep(500);
} catch (InterruptedException e) {
e.printStackTrace();
}
rd.request();
}
}
}).start();
//3s后，3号机器加入清单
Thread.currentThread().sleep(3000);
rd.addnode("192.168.0.3");
//3s后，当前服务节点被移除
Thread.currentThread().sleep(3000);
rd.remove("192.168.0.2");
}
}
Initialized as 1, 2, the two are not polled in sequence, but appear randomly.
3 Join the service node list.
After removing 2, only 1 and 3 are left, and the two are still random and out of order.
4. Source address hash (Hash) algorithm
1 Overview
Make a hash value for the currently accessed ip address, and the same key will be routed to the same machine. The scenario is common in a distributed cluster environment, request routing and session persistence when users log in.
2. Java implements address hash algorithm
Using HashMap can realize the service of requesting the value to the corresponding node, and the time complexity of the search is o(1). Fix an algorithm and map the request to the key.
For example, at the end of the source ip of the request, take the remainder according to the number of machines as the key:
public class Hash {
// 所有服务ip
ArrayList<String> ips ;
//初始化hash类，多个服务器ip用逗号隔开
public Hash(String nodeNames){
System.out.println("init list : "+nodeNames);
String[] nodes = nodeNames.split(",");
//初始化服务器列表，长度取机器数
ips = new ArrayList<>(nodes.length);
for (String node : nodes) {
ips.add(node);
}
}
//添加节点，注意，添加节点会造成内部Hash重排，思考为什么呢？？？
//这是个问题！在一致性hash中会进入详细探讨
void addnode(String nodeName){
System.out.println("add node : "+nodeName);
ips.add(nodeName);
}
//移除
void remove(String nodeName){
System.out.println("remove node : "+nodeName);
ips.remove(nodeName);
}
//映射到key的算法，这里取余数做下标
private int hash(String ip){
int last = Integer.valueOf(ip.substring(ip.lastIndexOf(".")+1,ip.length()));
return last % ips.size();
}
//请求
//注意，这里和来访ip是有关系的，采用一个参数，表示当前的来访ip
void request(String ip){
//下标
int i = hash(ip);
System.out.println(ip+"-->"+ips.get(i));
}
public static void main(String[] args) {
Hash hash = new Hash("192.168.0.1,192.168.0.2");
for (int i = 1; i < 10; i++) {
//模拟请求的来源ip
String ip = "192.168.0."+ i;
hash.request(ip);
}
hash.addnode("192.168.0.3");
for (int i = 1; i < 10; i++) {
//模拟请求的来源ip
String ip = "192.168.0."+ i;
hash.request(ip);
}
hash.remove("192.168.0.2");
for (int i = 1; i < 10; i++) {
//模拟请求的来源ip
String ip = "192.168.0."+ i;
hash.request(ip);
}
}
}
After initialization, there are only 1, 2, and the subscript is the remainder of the last ip. After running multiple times, the responding machine remains unchanged, and the session is maintained.
3 After joining, re-hash, the machine distribution changes.
After 2 is removed, the request that originally hashed to 2 is relocated to 3 response.
3. Consistent hashing
The source address hash algorithm allows certain requests to fall on the corresponding server. This solves the problem of session information retention.
At the same time, the standard hash, if the number of machine nodes changes. Then the request will be re-hash, breaking the original design intention, how to solve it? The answer is consistent hash.
(1) Principle
Taking 4 machines as an example, the algorithm of consistent hash is as follows:
first find the hash value of each server, and configure it on the circle of 0~ 232
; then use the same method to find the hash value of the stored data key The Greek value is also mapped on the circle;
search clockwise from the position where the data is mapped to, and save the data to the first server found;
if the maximum value is still not found, take the first one. This is why the image is called a ring.
Adding a node:
the principle of deleting a node is the same
(2) Characteristics
Monotonicity: Monotonicity means that if some requests have been assigned to the corresponding server for processing through hashing, and a new server is added to the system, it should be ensured that the original request can be mapped to the original Or go to a new server instead of being mapped to other original servers.
Dispersion (Spread): In a distributed environment, the client may only know a part of the server when requesting, then the two clients see different parts and think that what they see is a complete hash ring, then the problem comes , the same key may be routed to different servers. Take the above picture as an example, add client1 to see 1,4; client2 to see 2,3; then the key between 2-4 will be repeatedly mapped to 3,4 by the two clients. Dispersion reflects the severity of the problem.
Balance (Balance): Balance means that the request after the client hash should be able to be distributed to different servers. Consistent hash can be distributed as much as possible, but it cannot guarantee that the number of requests processed by each server is exactly the same. This deviation is called hash skew. If the node distribution algorithm design is not reasonable, then the balance will be greatly affected.
(3) Optimization
Adding virtual nodes can optimize the hash algorithm, making segmentation and distribution more refined. That is to say, there are actually m machines, but they are expanded by n times, and m*n machines are placed on the ring, then after equalization, the distribution of key segments will be more refined.
(4) Java implements consistent hash algorithm
import java.util.Random;
import java.util.SortedMap;
import java.util.TreeMap;
/**
* 不带虚拟节点的一致性Hash算法
*/
public class ConsistentHashingWithoutVirtualNode {
//服务器列表
private static String[] servers = {
"192.168.0.0", "192.168.0.1",
"192.168.0.2", "192.168.0.3", "192.168.0.4" };
//key表示服务器的hash值，value表示服务器
private static SortedMap<Integer, String> serverMap = new TreeMap<Integer, String>();
static {
for (int i=0; i<servers.length; i++) {
int hash = getHash(servers[i]);
//理论上，hash环的最大值为2^32
//这里为做实例，将ip末尾作为上限也就是254
//那么服务器是0-4，乘以60后可以均匀分布到 0-254 的环上去
//实际的请求ip到来时，在环上查找即可
hash *= 60;
System.out.println("add " + servers[i] + ", hash=" + hash);
serverMap.put(hash, servers[i]);
}
}
//查找节点
private static String getServer(String key) {
int hash = getHash(key);
//得到大于该Hash值的所有server
SortedMap<Integer, String> subMap = serverMap.tailMap(hash);
if(subMap.isEmpty()){
//如果没有比该key的hash值大的，则从第一个node开始
Integer i = serverMap.firstKey();
//返回对应的服务器
return serverMap.get(i);
}else{
//第一个Key就是顺时针过去离node最近的那个结点
Integer i = subMap.firstKey();
//返回对应的服务器
return subMap.get(i);
}
}
//运算hash值
//该函数可以自由定义，只要做到取值离散即可
//这里取ip地址的最后一节
private static int getHash(String str) {
String last = str.substring(str.lastIndexOf(".")+1,str.length());
return Integer.valueOf(last);
}
public static void main(String[] args) {
//模拟5个随机ip请求
for (int i = 0; i < 5; i++) {
String ip = "192.168.0."+ new Random().nextInt(254);
System.out.println(ip +" ---> "+getServer(ip));
}
//将5号服务器加到2-3之间，取中间位置，150
System.out.println("add 192.168.0.5，hash=150");
serverMap.put(150,"192.168.0.5");
//再次发起5个请求
for (int i = 0; i < 5; i++) {
String ip = "192.168.0."+ new Random().nextInt(254);
System.out.println(ip +" ---> "+getServer(ip));
}
}
}
4 machines join the hash ring
to simulate the request, and accurately schedule it to the downstream node according to the hash value.
Add node 5 and set the key to 150
to initiate the request again
5. Weighted Round Robin (WRR) Algorithm
1 Overview
WeightRoundRobin, polling is just a mechanical rotation, and weighted polling makes up for the shortcomings of all machines being treated equally. On a polling basis, upon initialization, the machine carries one 比重.
2. Java implements weighted round robin algorithm
Maintain a linked list, and each machine occupies a different number according to its weight. When polling, the weight is heavy, and the number is large, and the number of fetches will naturally increase. For example: a, b, c three machines, the weights are 4, 2, 1 respectively, after ranking will be a, a, a, a, b, b, c, each request, from the list in order Take the node, and then take the next one when you request it next time. When you get to the end, start over.
But there is a problem: the machines are not evenly distributed, and clusters appear...
Solution: In order to solve the problem of machine smoothness, a smooth weighted round-robin algorithm is used in the source code of nginx. The rules are as follows:
each node has two weights, weight and currentWeight. Weight is always the same as the configuration value. The current keeps changing;
the change rule is as follows: select all current += weight before selection, select the response with the largest current, and let its current -= total after the response.
Statistics: a=4, b=2, c=1 and the distribution is smooth and balanced
public class WRR {
//所有节点的列表
ArrayList<Node> list ;
//总权重
int total;
//初始化节点列表
public WRR(String nodes){
String[] ns = nodes.split(",");
list = new ArrayList<>(ns.length);
for (String n : ns) {
String[] n1 = n.split("#");
int weight = Integer.valueOf(n1[1]);
list.add(new Node(n1[0],weight));
total += weight;
}
}
//获取当前节点
Node getCurrent(){
//执行前，current加权重
for (Node node : list) {
node.currentWeight += node.weight;
}
//遍历，取权重最高的返回
Node current = list.get(0);
int i = 0;
for (Node node : list) {
if (node.currentWeight > i){
i = node.currentWeight;
current = node;
}
}
return current;
}
//响应
void request(){
//获取当前节点
Node node = this.getCurrent();
//第一列，执行前的current
System.out.print(list.toString()+"---");
//第二列，选中的节点开始响应
System.out.print(node.name+"---");
//响应后，current减掉total
node.currentWeight -= total;
//第三列，执行后的current
System.out.println(list);
}
public static void main(String[] args) {
WRR wrr = new WRR("a#4,b#2,c#1");
//7次执行请求，看结果
for (int i = 0; i < 7; i++) {
wrr.request();
}
}
class Node{
int weight,currentWeight; // 权重和current
String name;
public Node(String name,int weight){
this.name = name;
this.weight = weight;
this.currentWeight = 0;
}
@Override
public String toString() {
return String.valueOf(currentWeight);
}
}
}
6. Weighted Random (WR) Algorithm
1 Overview
WeightRandom, the machine is randomly screened, but a set of weighted values is made. According to different weights, the probability of selection is different. In this concept, randomness can be considered as a special case of equal weights.
2. Java implements weighted random algorithm
The design idea is still the same. According to the weight value, different numbers of nodes are generated. After the nodes are queued, they are randomly obtained. The data structure here mainly involves random reading, so it is preferably an array.
The same as random is that it is also randomly screened for the array, the difference is that random is only one for each machine, and it becomes multiple after weighting.
public class WR {
//所有节点的列表
ArrayList<String> list ;
//初始化节点列表
public WR(String nodes){
String[] ns = nodes.split(",");
list = new ArrayList<>();
for (String n : ns) {
String[] n1 = n.split("#");
int weight = Integer.valueOf(n1[1]);
for (int i = 0; i < weight; i++) {
list.add(n1[0]);
}
}
}
void request(){
//下标，随机数，注意因子
int i = new Random().nextInt(list.size());
System.out.println(list.get(i));
}
public static void main(String[] args) {
WR wr = new WR("a#2,b#1");
for (int i = 0; i < 9; i++) {
wr.request();
}
}
}
Run 9 times, a and b appear alternately, a=6, b=3, satisfying the ratio of 2:1
Note! Since it is random, there is randomness, and it may not be strictly proportional every time it is executed. When the sample tends to infinity, the proportion is approximately accurate
Seven, the minimum number of connections (LC) algorithm
1 Overview
LeastConnections, that is, to count the number of connections of the current machine, and choose the least one to respond to new requests. The previous algorithm is based on the request dimension, while the minimum number of connections is based on the machine dimension.
2. Java implements the minimum number of connections algorithm
Define a link table to record the node id of the machine and the counter of the number of machine connections. Internally, the minimum heap is used for sorting, and the top node of the heap is the minimum number of connections when responding.
public class LC {
//节点列表
Node[] nodes;
//初始化节点，创建堆
// 因为开始时各节点连接数都为0，所以直接填充数组即可
LC(String ns){
String[] ns1 = ns.split(",");
nodes = new Node[ns1.length+1];
for (int i = 0; i < ns1.length; i++) {
nodes[i+1] = new Node(ns1[i]);
}
}
//节点下沉，与左右子节点比对，选里面最小的交换
//目的是始终保持最小堆的顶点元素值最小
//i:要下沉的顶点序号
void down(int i) {
//顶点序号遍历，只要到1半即可，时间复杂度为O(log2n)
while ( i << 1 < nodes.length){
//左子，为何左移1位？回顾一下二叉树序号
int left = i<<1;
//右子，左+1即可
int right = left+1;
//标记，指向 本节点，左、右子节点里最小的，一开始取i自己
int flag = i;
//判断左子是否小于本节点
if (nodes[left].get() < nodes[i].get()){
flag = left;
}
//判断右子
if (right < nodes.length && nodes[flag].get() > nodes[right].get()){
flag = right;
}
//两者中最小的与本节点不相等，则交换
if (flag != i){
Node temp = nodes[i];
nodes[i] = nodes[flag];
nodes[flag] = temp;
i = flag;
}else {
//否则相等，堆排序完成，退出循环即可
break;
}
}
}
//请求。非常简单，直接取最小堆的堆顶元素就是连接数最少的机器
void request(){
System.out.println("---------------------");
//取堆顶元素响应请求
Node node = nodes[1];
System.out.println(node.name + " accept");
//连接数加1
node.inc();
//排序前的堆
System.out.println("before:"+Arrays.toString(nodes));
//堆顶下沉
down(1);
//排序后的堆
System.out.println("after:"+Arrays.toString(nodes));
}
public static void main(String[] args) {
//假设有7台机器
LC lc = new LC("a,b,c,d,e,f,g");
//模拟10个请求连接
for (int i = 0; i < 10; i++) {
lc.request();
}
}
class Node{
//节点标识
String name;
//计数器
AtomicInteger count = new AtomicInteger(0);
public Node(String name){
this.name = name;
}
//计数器增加
public void inc(){
count.getAndIncrement();
}
//获取连接数
public int get(){
return count.get();
}
@Override
public String toString() {
return name+"="+count;
}
}
}
After initialization, the heap node values are all 0, that is, the number of connections to each machine is 0.
After the top connection of the heap, the heap sinks, the heap is reordered, and the minimum heap rule remains true.
8. Application Cases
1、nginx upstream
upstream frontend {
#源地址hash
ip_hash;
server 192.168.0.1:8081;
server 192.168.0.2:8082 weight=1 down;
server 192.168.0.3:8083 weight=2;
server 192.168.0.4:8084 weight=3 backup;
server 192.168.0.5:8085 weight=4 max_fails=3 fail_timeout=30s;
}
ip_hash: the source address hash algorithm
down: means that the current server does not participate in the load temporarily
weight: the weighting algorithm, the default is 1, the greater the weight, the greater the weight of the load.
backup: backup machine, only when all other non-backup machines are down or busy, then request the backup machine.
max_fails: the maximum number of failures, the default value is 1, here is 3, that is, a maximum of 3 attempts
fail_timeout: the timeout period is 30 seconds, the default value is 10s.
Notice! weight and backup cannot be used together with the ip_hash keyword.
2、springcloud ribbon IRule
#设置负载均衡策略 eureka‐application‐service为调用的服务的名称
eureka‐application‐service.ribbon.NFLoadBalancerRuleClassName=com.netflix.loadbalancer.RandomRule
RoundRobinRule: Polling
RandomRule: Random
AvailabilityFilteringRule: First filter out services that are in the circuit breaker trip state due to multiple access failures, and services with concurrent connections exceeding the threshold, and then poll the remaining services WeightedResponseTimeRule: According to the average response
time Calculate the weight of all services, the faster the response time, the greater the weight of the service. If the statistical information is insufficient at the start, use the RoundRobinRule strategy. When the statistical information is sufficient, it will switch to this strategy. RetryRule:
First follow the RoundRobinRule strategy. If the service fails to be obtained, retry within the specified time to obtain available services.
BestAvailableRule: Yes First filter out services that are in the state of circuit breaker tripping due to multiple access failures, and then select a service with the smallest concurrency. ZoneAvoidanceRule
: The default rule comprehensively judges the performance of the area where the server is located and the availability of the server
3. Dubbo load balancing
Use Service annotations
@Service(loadbalance = "roundrobin",weight = 100)
RandomLoadBalance: Random, this method is dubbo's default load balancing strategy
RoundRobinLoadBalance: Polling
LeastActiveLoadBalance: The minimum active times, the dubbo framework has customized a Filter to calculate the number of service calls
ConsistentHashLoadBalance: Consistent hash