Original link: http://www.iocoder.cn/zhishixingqiu/?vip\x26amp;amp;amp;mp

Put away

Click on "Taro Source" above and select " Set as Star "

Whether she waves before or after?

A wave that can wave is a good wave!

Update the article at 8:55 every day, losing a billion hair every day...

Source code boutique column

Source: jianshu.com/p/d9504fc0af4d

Preface
Some ideas for dealing with large traffic
Common methods of current limiting
Current limiting artifact: Guava RateLimiter
Current limiting in distributed scenarios

Preface

In actual projects, I have encountered an online peak of 5W+QPS, and also experienced a large traffic request of 10W+QPS under stress testing. The topic of this blog is mainly my own thinking about high concurrent flow control.

Some ideas for dealing with large traffic

“
First of all, let's talk about what is large traffic?

Large traffic, we are likely to emerge: TPS (transactions per second), QPS (requests per second), 1W+, 5W+, 10W+, 100W+... In fact, there is no absolute number. If this amount causes system pressure and affects system performance, then this amount can be called a large flow.

Secondly, what are some common methods to deal with large traffic?

Caching : To put it bluntly, let the data enter the cache as soon as possible, close to the program, and don't access the DB frequently.

Downgrade : If it is not the core link, then downgrade the service. To use an analogy, the current APPs are all about thousands of people. After getting the data, they will display a personalized ranking. If the traffic is large, this ranking can be downgraded!

Current limit : Everyone knows that in the morning rush hour of the Beijing subway, the subway station will do one thing, that is, current limit! The idea is very straightforward, that is, to limit the request within a certain range within a certain period of time, to ensure that the system is not overwhelmed, and to increase the throughput of the system as much as possible.

Note that sometimes, caching and downgrading cannot solve the problem. For example, e-commerce double eleven, user purchases, orders and other behaviors involve a large number of write operations, and it is the core link that cannot be downgraded. At this time, current limiting is more important.

So next, let’s focus on current limiting.

Common methods of current limiting

“
Common processing methods for current limiting include counters, sliding windows, leaky buckets, and tokens.

counter

“
The counter is a relatively simple current limiting algorithm with a wide range of uses. At the interface level, many places use this method to limit current. In a period of time, count, compare with the threshold, and clear the counter to 0 at the critical point in time.

Counter idea

Code example

Counter code implementation

“
It should be noted here that there is a critical point in time. For example, there is no user request during the period from 12:01:00 to 12:01:58, and then 100 requests are issued at the instant of 12:01:59, OK, and then at the instant of 12:02:00 100 more requests were issued at that time. Here you should be able to feel that at this critical point, a large number of requests from malicious users may be accepted, even exceeding the expected tolerance of the system.

Sliding window

“
**Due to the critical point defect of the counter, a sliding window algorithm was later developed to solve it.

**

Sliding window schematic

“
Sliding window means to divide the fixed time slices and move them as time passes, so that the critical point of the counter is cleverly avoided. That is to say, these fixed number of movable grids will be counted to determine the threshold, so the number of grids affects the accuracy of the sliding window algorithm.

Leaky bucket

“
Although the sliding window effectively avoids the critical point of time, there is still the concept of time slice, and the leaky bucket algorithm is more advanced in this respect than the sliding window.

There is a fixed bucket, the rate of water inflow is uncertain, but the rate of water out is constant, and it will overflow when the water is full.

Leaky bucket algorithm idea

Code

Leaky bucket code implementation

Token bucket

“
Note that the water flow rate of the leaky bucket is constant, which means that if the instantaneous flow is large, most of the requests will be discarded (the so-called overflow). In order to solve this problem, the token bucket has been improved.

Principle of Token Bucket

“
The speed of generating tokens is constant, and there is no speed limit for requesting tokens. This means that, in the face of instantaneous large traffic, the algorithm can request a large number of tokens in a short period of time, and the process of obtaining tokens is not very expensive. (There is a bit of production token, the meaning of consumption token)

Whether it is rejected for the token bucket not being able to get the token, or the leaking bucket is full and overflowing, it is to ensure the normal use of most of the traffic, and a small part of the traffic is sacrificed. This is reasonable. If a small part of the traffic needs to be guaranteed, it may cause the system to reach its limit and hang up.

Code

Token bucket code implementation

Current limiting artifact: Guava RateLimiter

“
Guava is not only powerful in terms of collections, caching, asynchronous callbacks, etc. (you can refer to the blogger's "Happy Programming with Google Guava"), but it also encapsulates the current limiting API for us!

Guava RateLimiter is based on the token bucket algorithm. We only need to tell the RateLimiter system to limit the QPS, then RateLimiter will put tokens into the bucket at this speed, and then when requesting, obtain permission from RateLimiter through the tryAcquire() method (let brand).

Code example

RateLimiter

Current limiting in distributed scenarios

“
Some of the current limiting methods mentioned above are for a single machine. In fact, in most scenarios, the single-machine current limit is sufficient. Distributed lower current limit methods often require a combination of multiple technologies, such as Nginx+Lua, Redis+Lua, etc. This article mainly discusses the current limit of a single machine, and I will not introduce the current limit in a distributed scenario in detail here.

In a word, let the traffic of the system be queued and limited in the queue first, and don't let the traffic directly hit the system.

Well, here, this article is over!

good Morning!

A good day has begun, go to work!

Welcome to join my knowledge planet to discuss architecture and exchange source code. To join, press and hold the QR code below :

The source code has been updated in Knowledge Planet and the analysis is as follows:

Recently updated the "Taro SpringBoot 2.X Introduction" series, there are more than 20 articles, covering MyBatis, Redis, MongoDB, ES, sub-database sub-table, read-write separation, SpringMVC, Webflux, permissions, WebSocket, Dubbo, RabbitMQ, RocketMQ , Kafka, performance testing, etc.

Provide SpringBoot samples with nearly 3W lines of code, and e-commerce microservice projects with more than 4W lines of code.

Obtaining method: Click " Watching ", follow the official account and reply to 666 to receive, more content will be provided one after another.

Brothers, Susan Chan a, point a praise ! ????

When I was asked about high concurrent flow control during the interview, my face turned green. . .