Current limit realization 2

Introduction

Previous article limiting achieve 1 already describes the three current-limiting scheme

Random refusal
Counter method
Current limit based on sliding time window

The remaining few were originally intended to be finished immediately, but I didn't expect three months to pass, which was very embarrassing. This time, the following two algorithms are mainly implemented

Token bucket algorithm
Funnel algorithm

The specific implementation of the algorithm can be viewed on github https://github.com/shidawuhen/asap/tree/master/controller/limit

Let’s first introduce these two algorithms.

Token bucket algorithm

Algorithm description

Token Bucket: It is the most commonly used algorithm in network traffic shaping (Traffic Shaping) and rate limiting (Rate Limiting). The schematic diagram of the token bucket algorithm is as follows:

The general process is:

a. Put tokens into the token bucket at a specific rate

b. According to the preset matching rules, the packets are classified first, and the packets that do not meet the matching rules do not need to be processed by the token bucket and sent directly;

c. Packets that meet the matching rules need to be processed by the token bucket. When there are enough tokens in the bucket, the message can continue to be sent, and the amount of tokens in the token bucket is reduced correspondingly according to the length of the message;

d. When there are insufficient tokens in the token bucket, the message cannot be sent. The message can be sent only when a new token is generated in the bucket. This can limit the flow of messages to only less than or equal to the rate of token generation, achieving the purpose of limiting the flow.

The fixed-size token bucket can continuously generate tokens at a constant rate. If the token is not consumed, or the rate of consumption is less than the rate of generation, the tokens will continue to increase until the bucket is filled. The tokens generated later will overflow from the bucket. The maximum number of tokens that can be stored in the final bucket will never exceed the size of the bucket.

The token bucket can allow bursts because the token bucket generally has two values, one is the bucket capacity, and the other is the amount of tokens placed in a unit of time. If the bucket capacity is greater than the token quantity delivered per unit time, and the consumption per unit time is less than the quantity delivered, the number of tokens will eventually reach the maximum bucket capacity. If a large number of requests arrive at this time, all tokens will be consumed, realizing the effect of allowing bursts.

Algorithm implementation

The algorithm has several core points

Because you want to update the number of tokens, you need to lock
There are two ways to put tokens into the bucket regularly, one is to start a goroutine and put it regularly, and the other is to put it according to the situation when judging whether there are enough tokens. This time the implementation uses the second method, and the whole architecture will be simpler.

package limit

import (
   "github.com/gin-gonic/gin"
   "net/http"
   "sync"
   "time"
)

// @Tags limit
// @Summary 令牌桶拒流
// @Produce  json
// @Success 200 {string} string "成功会返回ok"
// @Failure 502 "失败返回reject"
// @Router /limit/tokenreject [get]
type TokenBucket struct {
    
    
   rate         int64 //固定的token放入速率, r/s
   capacity     int64 //桶的容量
   tokens       int64 //桶中当前token数量
   lastTokenSec int64 //桶上次放token的时间戳 s

   lock sync.Mutex
}

func (l *TokenBucket) Allow() bool {
    
    
   l.lock.Lock()
   defer l.lock.Unlock()

   now := time.Now().Unix()
   l.tokens = l.tokens + (now-l.lastTokenSec)*l.rate // 先添加令牌
   if l.tokens > l.capacity {
    
    
      l.tokens = l.capacity
   }
   l.lastTokenSec = now
   if l.tokens > 0 {
    
    
      // 还有令牌，领取令牌
      l.tokens--
      return true
   } else {
    
    
      // 没有令牌,则拒绝
      return false
   }
}

func (l *TokenBucket) Set(r, c int64) {
    
    
   l.rate = r
   l.capacity = c
   l.tokens = 0
   l.lastTokenSec = time.Now().Unix()
}

func CreateTokenBucket()*TokenBucket{
    
    
   t := &TokenBucket{
    
    }
   t.Set(1,5)
   return t
}

var tokenBucket *TokenBucket = CreateTokenBucket()

func TokenReject(c *gin.Context) {
    
    
   //fmt.Println(tokenBucket.tokens)
   if !tokenBucket.Allow() {
    
    
      c.String(http.StatusBadGateway, "reject")
      return
   }
   c.String(http.StatusOK, "ok")
}

Leaky bucket algorithm

Algorithm description

When the leaky bucket is used as a metering tool (The Leaky Bucket Algorithm as a Meter), it can be used for traffic shaping (Traffic Shaping) and flow control (TrafficPolicing). The description of the leaky bucket algorithm is as follows:

A leaky bucket with a fixed capacity, which flows out water droplets at a constant and fixed rate;
If the bucket is empty, no water droplets need to flow out;
Water droplets can be poured into the leaky bucket at any rate;
If the inflowing water droplets exceed the capacity of the bucket, the inflowing water droplets are overflowed (discarded), and the leaky bucket capacity remains unchanged.

The leaky bucket method of current limiting is easy to understand. Suppose we have a bucket that drops a drop of water at a fixed rate. No matter how many requests and how big the request rate is, it will flow out at a fixed rate. Corresponding to the system, it is based on a fixed rate. The rate at which requests are processed.

The schematic diagram is as follows:

Algorithm implementation

After consulting the relevant information, there are three main algorithm implementations. After studying these kinds of implementations, I suspect that I have misunderstood it, and I feel that it is not convenient to use the counting refusal. If my understanding is wrong, you can also tell me.

The three ways are:

Variants of token bucket algorithm

The size of the bucket is the maximum amount that can flow out per unit time, this kind of not written.

Counting reflux variant

This method sets the available space to the initial value at the specified time.

package limit

import (
	"fmt"
	"github.com/gin-gonic/gin"
	"net/http"
	"sync"
	"time"
)

type LeakyBucket struct {
    
    
	// 容量
	capacity  int64
	// 剩余大小
	remaining int64
	// 下一次的重置容量时间
	reset     time.Time
	// 重置容量时间间隔
	rate      time.Duration
	mutex     sync.Mutex
}
func (b *LeakyBucket) Allow() bool {
    
    
	b.mutex.Lock()
	defer b.mutex.Unlock()
	if time.Now().After(b.reset) {
    
     // 需要重置
		b.reset = time.Now().Add(b.rate) // 更新时间
		b.remaining = b.capacity // 重置剩余容量
	}
	fmt.Println(b.remaining)
	if b.remaining > 0 {
    
     // 判断是否能过
		b.remaining--
		return true
	}
	return false
}

func (b *LeakyBucket) Set(r time.Duration, c int64) {
    
    
	b.rate = r
	b.capacity = c
	b.remaining = c
	b.reset = time.Now().Add(b.rate)
}

func CreateLeakyBucket(r time.Duration,c int64) *LeakyBucket {
    
    
	t := &LeakyBucket{
    
    }
	t.Set(r, c)
	return t
}

var leakyBucket *LeakyBucket = CreateLeakyBucket(time.Second*2,10)
func LeakyReject(c *gin.Context) {
    
    
	if !leakyBucket.Allow() {
    
    
		c.String(http.StatusBadGateway, "reject")
		return
	}
	c.String(http.StatusOK, "ok")
}

True fixed rate

The implementation of this algorithm is based on the open source github.com/uber-go/ratelimit of the uber team .

This implementation guarantees that if there are a large number of requests, each request will be executed at a specified time interval. If you set 1s to process 100 requests, then one will be processed every 10ms.

If there is no request for a long time, the request will still be processed in a short time. Of course, this situation can be easily repaired, and you can think about how to modify it.

package limit

import (
   "fmt"
   "github.com/andres-erbsen/clock"
   "github.com/gin-gonic/gin"
   "net/http"
   "sync"
   "time"
)

//真固定速率
type Clock interface {
    
    
   Now() time.Time
   Sleep(time.Duration)
}

type limiter struct {
    
    
   sync.Mutex               // 锁
   last       time.Time     // 上一次的时刻
   sleepFor   time.Duration // 需要等待的时间
   perRequest time.Duration // 每次的时间间隔
   maxSlack   time.Duration // 最大的富余量
   clock      Clock         // 时钟
}

// Take 会阻塞确保两次请求之间的时间走完
// Take 调用平均数为 time.Second/rate.
func (t *limiter) Take() time.Time {
    
    
   t.Lock()
   defer t.Unlock()

   now := t.clock.Now()

   // 如果是第一次请求就直接放行
   if t.last.IsZero() {
    
    
      t.last = now
      return t.last
   }

   // sleepFor 根据 perRequest 和上一次请求的时刻计算应该sleep的时间
   // 由于每次请求间隔的时间可能会超过perRequest, 所以这个数字可能为负数，并在多个请求之间累加
   t.sleepFor += t.perRequest - now.Sub(t.last)
   fmt.Println(t.sleepFor)
   // 我们不应该让sleepFor负的太多，因为这意味着一个服务在短时间内慢了很多随后会得到更高的RPS。
   if t.sleepFor < t.maxSlack {
    
    
      t.sleepFor = t.maxSlack
   }

   // 如果 sleepFor 是正值那么就 sleep
   if t.sleepFor > 0 {
    
    
      t.clock.Sleep(t.sleepFor)
      t.last = now.Add(t.sleepFor)
      t.sleepFor = 0
   } else {
    
    
      t.last = now
   }
   return t.last
}

func NewLimiter(rate int) *limiter {
    
    
   l := &limiter{
    
    
      perRequest: time.Second / time.Duration(rate),
      maxSlack:   -10 * time.Second / time.Duration(rate),
   }

   if l.clock == nil {
    
    
      l.clock = clock.New()
   }
   return l
}

var rl = NewLimiter(100) // per second,每秒100个请求
func LeakyRejectFixedRate(c *gin.Context) {
    
    
   prev := time.Now()
   for i := 0; i < 10; i++ {
    
    
      now := rl.Take()
      fmt.Println(i, now.Sub(prev))
      prev = now
   }
   c.String(http.StatusOK, "ok")
}

The results of the demonstration are as follows:

to sum up

After learning the token bucket and leaky bucket algorithm, combined with the specific scenarios of the work in the past few years, I feel that the practical value of the token bucket algorithm is greater.

The following is a comparison of token bucket and leaky bucket:

The token bucket adds tokens to the bucket at a fixed rate. Whether the request is processed depends on whether the tokens in the bucket are sufficient. When the number of tokens is reduced to zero, new requests are rejected;
Leaky buckets flow out requests at a constant fixed rate, and the rate of incoming requests is arbitrary. When the number of incoming requests accumulates to the leaky bucket capacity, the new incoming requests are rejected;
The token bucket limits the average inflow rate (allowing burst requests, as long as there are tokens can be processed, supporting 3 tokens and 4 tokens at a time), and allows a certain degree of burst traffic;
The leaky bucket limits the constant outflow rate (that is, the outflow rate is a fixed constant value, for example, the outflow rate is 1, and cannot be 1 once and 2 again next time), thereby smoothing the burst inflow rate;
The token bucket allows a certain degree of burst, and the main purpose of the leaky bucket is to smooth the inflow rate;
The two algorithms can be implemented in the same way, but the directions are opposite, and the current limiting effect obtained for the same parameters is the same.

Finally, I will show you a comparison of various current limiting algorithms.

data

At last

If you like my article, you can follow my official account (Programmer Mala Tang)

My personal blog is: https://shidawuhen.github.io/

Review of previous articles:

technology

study notes

Thinking