Current limiting, also known as flow control (flow control), usually refers to limiting the number of concurrent requests that reach the system. This article lists common current limiting strategies, and uses the gin framework as an example to demonstrate how to add current limiting components to the project.
Limiting
Current limiting is also called flow control (flow control), and usually refers to limiting the number of concurrent requests that reach the system.
We also often encounter flow restriction scenarios in our lives. For example, a certain scenic spot limits the number of tourists who enter the scenic spot every day to 80,000; Shahe Metro Station limits the number of passengers entering the station at the same time by queuing outside the station one by one during the morning peak. Quantity etc.
Although the current limit will affect the user experience of some users, it can report the stability of the system to a certain extent without crashing (everyone has no user experience).
There are many similar business scenarios on the Internet that require current limiting, such as the spike in the e-commerce system, breaking hot news on Weibo, the Double Eleven shopping festival, 12306 ticket grabbing and so on. The amount of user requests in these scenarios usually increases sharply, far exceeding the normal amount of requests. At this time, if you do not add any restrictions, it will easily break the back-end service and affect the stability of the service.
In addition, the API services disclosed by some manufacturers usually limit the number of user requests. For example, the open platform of Baidu Maps will limit the number of user requests based on the user's payment status.
Commonly used current limiting strategies
Leaky bucket
The leaky bucket method of current limiting is easy to understand. Suppose we have a bucket that drops a drop of water at a fixed rate. No matter how many requests and how big the request rate is, it will flow out at a fixed rate. Corresponding to the system, it is based on a fixed rate. The rate at which requests are processed.
The key point of the leaky bucket method is that the leaky bucket always runs at a fixed rate, but it cannot handle scenarios with a large number of burst requests. After all, in some scenarios, we may need to improve the processing efficiency of the system instead of blindly. Process requests at a fixed rate.
Regarding the implementation of leaky buckets, the uber team has an open source github.com/uber-go/ratelimit library. The method of using this library is relatively simple, the Take() method will return the time of the next dripping of the leaky bucket.
import (
"fmt"
"time"
"go.uber.org/ratelimit")func main() {
rl := ratelimit.New(100)
prev := time.Now()
for i := 0; i < 10; i++ {
now := rl.Take()
fmt.Println(i, now.Sub(prev))
prev = now }
}
Its source code implementation is also relatively simple, here is a general overview of the key points, interested students can go and see the complete source code for themselves.
The limiter is an interface type that requires a Take()method to be implemented :
type Limiter interface {
Take() time.Time}
The structure that implements the limiter interface is defined as follows. Here you can focus on the following maxSlackfields, which Take()are processed in the following methods.
type limiter struct {
sync.Mutex
last time.Time
sleepFor time.Duration
perRequest time.Duration
maxSlack time.Duration
clock Clock
}
limiterLimiterThe Take()method content of the structure to implement the interface is as follows:
func (t *limiter) Take() time.Time {
t.Lock()
defer t.Unlock()
now := t.clock.Now()
if t.last.IsZero() {
t.last = now return t.last }
t.sleepFor += t.perRequest - now.Sub(t.last)
if t.sleepFor < t.maxSlack {
t.sleepFor = t.maxSlack }
if t.sleepFor > 0 {
t.clock.Sleep(t.sleepFor)
t.last = now.Add(t.sleepFor)
t.sleepFor = 0
} else {
t.last = now }
return t.last}
The above code calculates the time that the current request needs to be blocked according to the interval time of each request and the time of the last request. It sleepForshould be noted that sleepForthe value may be negative, after two visits with a long interval. Will cause a large number of requests to be released subsequently, so the code has a special optimization process for this scenario. The New()function of creating the limiter will maxSlackset the initial value, and WithoutSlackthe default value can also be canceled through this Option.
func New(rate int, opts ...Option) Limiter {
l := &limiter{
perRequest: time.Second / time.Duration(rate),
maxSlack: -10 * time.Second / time.Duration(rate),
}
for _, opt := range opts {
opt(l)
}
if l.clock == nil {
l.clock = clock.New()
}
return l}
Token bucket
The principle of the token bucket is similar to the leaky bucket. The token bucket puts tokens into the bucket at a fixed rate, and as long as the token can be taken out of the bucket, it can pass through. The token bucket supports rapid processing of burst traffic.
For the scenario where the token cannot be obtained from the bucket, we can choose to wait or directly reject and return.
For the Go language implementation of the token bucket, you can refer to the github.com/juju/ratelimit library. This library supports multiple token bucket modes and is relatively simple to use.
How to create a token bucket:
func NewBucket(fillInterval time.Duration, capacity int64) *Bucketfunc NewBucketWithQuantum(fillInterval time.Duration, capacity, quantum int64) *Bucketfunc NewBucketWithRate(rate float64, capacity int64) *Bucket
The method to withdraw the token is as follows:
func (tb *Bucket) Take(count int64) time.Durationfunc (tb *Bucket) TakeAvailable(count int64) int64func (tb *Bucket) TakeMaxDuration(count int64, maxWait time.Duration) (time.Duration, bool)func (tb *Bucket) Wait(count int64)func (tb *Bucket) WaitMaxDuration(count int64, maxWait time.Duration) bool
Although it is a token bucket, we don’t really need to generate tokens and put them in the bucket. We just need to calculate it every time we fetch tokens, whether there are enough tokens at present, and the specific calculation method It can be summarized as the following formula:
当前令牌数 = 上一次剩余的令牌数 + (本次取令牌的时刻-上一次取令牌的时刻)/放置令牌的时间间隔 * 每次放置的令牌数
The source code for calculating the number of tokens in the library github.com/juju/ratelimit is as follows:
func (tb *Bucket) currentTick(now time.Time) int64 {
return int64(now.Sub(tb.startTime) / tb.fillInterval)}
func (tb *Bucket) adjustavailableTokens(tick int64) {
if tb.availableTokens >= tb.capacity {
return
}
tb.availableTokens += (tick - tb.latestTick) * tb.quantum if tb.availableTokens > tb.capacity {
tb.availableTokens = tb.capacity }
tb.latestTick = tick return}
TakeAvailable()The source code of the key part of the function to obtain the token is as follows:
func (tb *Bucket) takeAvailable(now time.Time, count int64) int64 {
if count <= 0 {
return 0
}
tb.adjustavailableTokens(tb.currentTick(now))
if tb.availableTokens <= 0 {
return 0
}
if count > tb.availableTokens {
count = tb.availableTokens }
tb.availableTokens -= count return count}
You can also see from the code that the implementation of the token bucket is not very complicated.
Use current limiting middleware in the gin framework
In projects built by the gin framework, we can define current limiting components as middleware.
Here, the token bucket is used as a current-limiting strategy, and a current-limiting middleware is written as follows:
func RateLimitMiddleware(fillInterval time.Duration, cap int64) func(c *gin.Context) {
bucket := ratelimit.NewBucket(fillInterval, cap)
return func(c *gin.Context) {
if bucket.TakeAvailable(1) < 1 {
c.String(http.StatusOK, "rate limit...")
c.Abort()
return
}
c.Next()
}}
Regarding the registration location of the current-limiting middleware, we can register it to different locations according to different current-limiting strategies, for example:
-
If you want to limit the flow of the entire site, you can register as a global middleware.
-
If it is a certain group of routes that require current limiting, then just register the current limiting middleware to the corresponding routing group.
