Load balancing algorithm-polling

Load balancing algorithm-polling

---

table of Contents

1. Overview
2. Simple polling
3. Weighted polling
4. Smooth weighted polling

---

1 Overview

1. In a distributed system, in order to achieve load balancing, load scheduling algorithms, such as Nginx and RPC service discovery, are inevitably involved. Common load balancing algorithms include polling, source address hash, and the least number of connections. Polling is the simplest and most widely used algorithm.
2. Three common round-robin scheduling algorithms are simple round-robin, weighted round-robin, and smooth weighted round-robin. The following four services will be used to illustrate the polling scheduling process in detail.

Service instance	Weights
192.168.10.1:2202	1
192.168.10.2:2202	2
192.168.10.3:2202	3
192.168.10.4:2202	4

---

2. Simple polling

1. Simple polling is the simplest polling algorithm, but because it does not support configuration load, it has fewer applications.

1. Algorithm description

1. Assuming there are N instances S = {S1, S2, …, Sn}, the indicator variable currentPos represents the currently selected instance ID, which is initialized to -1. The algorithm can be described as:
   1. Schedule to the next instance;
   2. If all instances have been scheduled once, start from the beginning;
   3. Repeat steps 1 and 2 for each scheduling;

request	currentPos	Selected instance
1	0	192.168.10.1:2202
2	1	192.168.10.2:2202
3	2	192.168.10.3:2202
4	3	192.168.10.4:2202
5	0	192.168.10.1:2202

2. Code implementation

type Round struct {
    
    
	curIndex int
	rss      []string
}

func (r *Round) Add(params ...string) error {
    
    
	if len(params) == 0 {
    
    
		return errors.New("至少需要1个参数")
	}
	r.rss = append(r.rss, params...)
	return nil
}
func (r *Round) Next() (string, error) {
    
    
	if len(r.rss) == 0 {
    
    
		return "", errors.New("不存在参数")
	}
	curElement := r.rss[r.curIndex]
	r.curIndex = (r.curIndex + 1) % len(r.rss)
	return curElement, nil
}

3. Advantages and disadvantages

1. In actual applications, the same service will be deployed to different hardware environments, and different performances will occur. If you directly use the simple round-robin scheduling algorithm to give each service instance the same load, then there will inevitably be a waste of resources. Therefore, in order to avoid this situation, some people have proposed the following weighted polling algorithm.

---

2. Weighted polling

1. The weighted round-robin algorithm introduces the "weight" value and improves the simple round-robin algorithm. The weight of the instance load can be configured according to the hardware performance, so as to achieve the rational use of resources.

1. Algorithm description

1. Suppose there are N instances S = {S1, S2, …, Sn}, weight W = {W1, W2, …, Wn}, indicator variable currentPos represents the currently selected instance ID, initialized to -1; variable currentWeight represents current weight , The initial value is max(S); max(S) represents the maximum weight value of N instances, and gcd(S) represents the greatest common divisor of the weight of N instances.

2. The algorithm can be described as:

   1. From the last instance of scheduling, traverse every subsequent instance;
   2. If all instances have been traversed once, reduce currentWeight to currentWeight-gcd(S), and start traversing from the beginning; if currentWeight is less than or equal to 0, reset to max(S);
   3. It ends when the weight of the traversed instance is equal to or greater than currentWeight, and the instance is the instance that needs to be scheduled;
   4. Repeat steps 1, 2, and 3 for each scheduling;

3. For example, for the above 4 services, the maximum weight max(S) is 4, and the greatest common divisor gcd(S) is 1. The scheduling process is as follows:

request	currentPos	currentWeight	Selected instance
1	3	4	192.168.10.4:2202
2	2	3	192.168.10.3:2202
3	3	3	192.168.10.4:2202
4	1	2	192.168.10.2:2202
…	…	…	….
9	2	1	192.168.10.3:2202
10	3	4	192.168.10.4:2202

2. Code implementation

var slaveDns = map[int]map[string]interface{
    
    }{
    
    
	0: {
    
    "connectstring": "root@tcp(172.16.0.164:3306)/shiqu_tools?charset=utf8", "weight": 2},
	1: {
    
    "connectstring": "root@tcp(172.16.0.165:3306)/shiqu_tools?charset=utf8", "weight": 4},
	2: {
    
    "connectstring": "root@tcp(172.16.0.166:3306)/shiqu_tools?charset=utf8", "weight": 8},
}

var last int = -1    //表示上一次选择的服务器
var cw int = 0       //表示当前调度的权值
var gcd int = 2      //当前所有权重的最大公约数 比如 2，4，8 的最大公约数为：2
var devCount int = 2 //当前机器数

func getDns() string {
    
    
	for {
    
    
		last = (last + 1) % len(slaveDns)
		if last == 0 {
    
    
			cw = cw - gcd
			if cw <= 0 {
    
    
				cw = getMaxWeight()
				if cw == 0 {
    
    
					return ""
				}
			}
		}

		if weight, _ := slaveDns[last]["weight"].(int); weight >= cw {
    
    
			return slaveDns[last]["connectstring"].(string)
		}
	}
}

func getMaxWeight() int {
    
    
	max := 0
	for _, v := range slaveDns {
    
    
		if weight, _ := v["weight"].(int); weight >= max {
    
    
			max = weight
		}
	}
	return max
}

func Add(addr string, weight int) {
    
    
	tmap := make(map[string]interface{
    
    })
	tmap["connectstring"] = addr
	tmap["weight"] = weight
	slaveDns[devCount] = tmap

	devCount = devCount + 1
	
	if devCount == 0 {
    
    
		gcd = weight
	} else {
    
    
		gcd = Gcd(gcd, weight)
	}
}

func Gcd(gcd int, weight int) int {
    
    
	for weight != 0 {
    
    
		gcd, weight = weight, gcd%weight
	}
	return gcd
}

3. Advantages and disadvantages

1. Although the weighted polling algorithm solves the resource utilization problem of simple polling by configuring instance weights, it still has a relatively obvious defect.
2. For example: service instance S = {a, b, c}, weight W = {5, 1, 1}, the sequence of instances generated by weighted round-robin scheduling is {a, a, a, a, a, b, c} , Then there will be 5 consecutive requests scheduled to instance a. In practice, this uneven load is not allowed, because continuous requests will suddenly increase the load of instance a, which may cause serious accidents.
3. In order to solve the defect of uneven weighted round-robin scheduling, a smooth weighted round-robin scheduling algorithm is proposed, which generates a more uniform scheduling sequence {a, a, b, a, c, a, a}.

---

4. Smooth weighted polling

1. Algorithm description

1. Suppose there are N instances S = {S1, S2, …, Sn}, configuration weight W = {W1, W2, …, Wn}, and effective weight CW = {CW1, CW2, …, CWn}. In addition to a configuration weight Wi for each instance i, there is also a current effective weight CWi, and CWi is initialized to Wi; the indicator variable currentPos represents the currently selected instance ID, which is initialized to -1; the sum of the configuration weights of all instances is weightSum;

2. Then, the scheduling algorithm can be described as:

   1. Initially, the current effective weight CWi of each instance i is the configuration weight Wi, and the configuration weight and weightSum are obtained;
   2. Select the instance with the largest current effective weight, subtract the weight and weightSum of all instances from the current effective weight CWi, and the variable currentPos points to this position;
   3. Add the configuration weight Wi to the current effective weight CWi of each instance i;
   4. Get the instance pointed to by the variable currentPos;
   5. Repeat the above steps 2, 3, 4 for each scheduling;

3. For the above three services, the configuration weight and weightSum are 7, and the scheduling process is as follows:

request	Current weight before selection	currentPos	Selected instance	Current weight after selection
1	{5, 1, 1}	0	192.168.10.1:2202	{-2, 1, 1}
2	{3, 2, 2}	0	192.168.10.1:2202	{-4, 2, 2}
3	{1, 3, 3}	1	192.168.10.2:2202	{1, -4, 3}
4	{6, -3, 4}	0	192.168.10.1:2202	{-1, -3, 4}
5	{4, -2, 5}	2	192.168.10.3:2202	{4, -2, -2}
6	{9, -1, -1}	0	192.168.10.1:2202	{2, -1, -1}
7	{7, 0, 0}	0	192.168.10.1:2202	{0, 0, 0}
8	{5, 1, 1}	0	192.168.10.1:2202	{-2, 1, 1}

4. The idea of ​​this round-robin scheduling algorithm was first proposed by the Nginx developers

2. Code implementation

type LoadBalance interface {
    
    
	//选择一个后端Server
	//参数remove是需要排除选择的后端Server
	Select(remove []string) *Server
	//更新可用Server列表
	UpdateServers(servers []*Server)
}

type Server struct {
    
    
	//主机地址
	Host string
	//主机名
	Name   string
	Weight int
	//主机是否在线
	Online bool
}

type Weighted struct {
    
    
	Server          *Server
	Weight          int
	CurrentWeight   int //当前机器权重
	EffectiveWeight int //机器权重
}

func (this *Weighted) String() string {
    
    
	return fmt.Sprintf("[%s][%d]", this.Server.Host, this.Weight)
}

type LoadBalanceWeightedRoundRobin struct {
    
    
	servers  []*Server
	weighted []*Weighted
}

func NewLoadBalanceWeightedRoundRobin(servers []*Server) *LoadBalanceWeightedRoundRobin {
    
    
	new := &LoadBalanceWeightedRoundRobin{
    
    }
	new.UpdateServers(servers)
	return new
}

func (this *LoadBalanceWeightedRoundRobin) UpdateServers(servers []*Server) {
    
    
	if len(this.servers) == len(servers) {
    
    
		for _, new := range servers {
    
    
			isEqual := false
			for _, old := range this.servers {
    
    
				if new.Host == old.Host && new.Weight == old.Weight && new.Online == old.Online {
    
    
					isEqual = true
					break
				}
			}
			if isEqual == false {
    
    
				goto build
			}
		}
		return
	}

build:
	log.Println("clients change")
	log.Println(this.servers)
	log.Println(servers)
	weighted := make([]*Weighted, 0)
	for _, v := range servers {
    
    
		if v.Online == true {
    
    
			w := &Weighted{
    
    
				Server:          v,
				Weight:          v.Weight,
				CurrentWeight:   0,
				EffectiveWeight: v.Weight,
			}
			weighted = append(weighted, w)
		}
	}
	this.weighted = weighted
	this.servers = servers
	log.Printf("weighted[%v]", this.weighted)
}

func (this *LoadBalanceWeightedRoundRobin) Select(remove []string) *Server {
    
    
	if len(this.weighted) == 0 {
    
    
		return nil
	}
	w := this.nextWeighted(this.weighted, remove)
	if w == nil {
    
    
		return nil
	}
	return w.Server
}

func (this *LoadBalanceWeightedRoundRobin) nextWeighted(servers []*Weighted, remove []string) (best *Weighted) {
    
    
	total := 0
	for i := 0; i < len(servers); i++ {
    
    
		w := servers[i]
		if w == nil {
    
    
			continue
		}
		isFind := false
		for _, v := range remove {
    
    
			if v == w.Server.Host {
    
    
				isFind = true
			}
		}
		if isFind == true {
    
    
			continue
		}

		w.CurrentWeight += w.EffectiveWeight
		total += w.EffectiveWeight
		if w.EffectiveWeight < w.Weight {
    
    
			w.EffectiveWeight++
		}

		if best == nil || w.CurrentWeight > best.CurrentWeight {
    
    
			best = w
		}
	}
	if best == nil {
    
    
		return nil
	}
	best.CurrentWeight -= total
	return best
}

func (this *LoadBalanceWeightedRoundRobin) String() string {
    
    
	return "WeightedRoundRobin"
}

3. Summary

1. Although the smooth weighted round-robin algorithm improves the shortcomings of the weighted round-robin algorithm scheduling, that is, the uneven distribution of the scheduling sequence and avoids the possibility of a sudden increase in the instance load, it still cannot dynamically perceive the load of each instance.
2. If the instance weight configuration is unreasonable or the system load is aggravated by some other reasons, smooth weighted polling cannot achieve load balancing for each instance, and a stateful scheduling algorithm is needed to complete it.