Interface limiting, service degradation, fusing

Interface limiting

Why limiting

Dealing with customer service

Such as web services, external API, this type of service are the following might cause the machine to be worn down

1. Users excessive growth (which is good)
2. Because some hot events (microblogging hot search)
3. Competitors reptiles
4. Malicious scalping

These cases are unpredictable, do not know there will be 10 times or 20 times the traffic when to come in, if you encounter such a situation, the expansion is no time, the elastic expansion is too late;

Internal RPC service

A service interface may be a plurality of BCDE service calls, when the service B burst traffic occurs, directly linked to the calling service A, causing A CDE can not provide service to service. When this has occurred, there are two solutions:

1. Each caller thread pool for resource isolation
2. Limiting the use of means of limiting each caller

Limiting the algorithm

Limiting single algorithm are:

1. counter
2. Token Bucket
3. Leaky Bucket

1, counter algorithm

Principle: The counter for current limiting somewhat simple and crude, generally limit the number of requests we can pass a second.

100 qps limiting example, the algorithm is the idea to realize a request comes in from the first start time, in the next 1s, a request to each, put the count is incremented, if the accumulated number reaches 100, then the subsequent all requests will be rejected. Wait until the end of the 1s, the recovery count to zero to start counting again.

Implementation: For each service call, you can add 1 to the counter and returns the latest value by comparing the latest and thresholds by AtomicLong.incrementAndGet () method.

Drawbacks: If I pre-1s 10ms per unit of time, has passed the 100 requests that the back of 990ms, can only watch helplessly reject the request, we call this phenomenon "thrusting phenomenon."

2, leaky bucket algorithm

In order to eliminate the "thrusting phenomenon" leaky bucket algorithms can be used for current limiting, leaky bucket algorithm name is very image.

Principle: a container internal algorithm, the funnel-like life used, when a request comes in, the equivalent of water poured into the funnel, and then flows slowly from the lower end of the uniform small mouth. No matter how much traffic above the outflow below the speed remains the same. If the container is full, the new incoming request will be discarded.

No matter how unstable the caller to the service, to limit the flow through leaky bucket algorithm, once every 10 milliseconds processing requests. Because processing speed is fixed, the speed of the incoming request is unknown, may suddenly come in many requests, the request did not have time to process it in the dustbin, since it is a barrel, there is definitely a capacity limit, if the bucket is full, new incoming request is discarded.

Implementation: In terms of algorithm, a queue may be prepared for storage request, obtained by a further thread pool periodic requests from the queue and executed, can acquire disposable multiple concurrent execution.

Drawbacks: Unable to deal with unexpected traffic for a short time.

3, the token bucket algorithm

In a sense, the token bucket algorithm is an improvement of the leaky bucket, bucket algorithm can be invoked to limit the rate request, the token bucket algorithm can also allow a degree of projections to limit the average rate of call while hair calls.

Principle: In the token bucket algorithm, there is a bucket used to store a fixed number of tokens. There is a mechanism algorithm, at a rate to put token bucket. Each time you need to get a token request to call, only to get a token, have a chance to continue, or choose to wait for the token selection available, or directly rejected.

This action is the release token is continuous, if the number of tokens in the bucket reaches the upper limit, the token is discarded, so there are cases where the bucket has a large number of available tokens, then the incoming request can be directly get a token to perform, such as setting qps to 100, after initialization is complete current limiter second, bucket, there were already 100 tokens, and then start the service is not completely good, and so when you start to complete the external services, the flow restrictor 100 can withstand instantaneous request. So, not only the token bucket, the request will be waiting, and finally at a rate equivalent to the implementation.

Realization of ideas: a queue can be prepared, for holding the token, further scheduled by a thread token pool into the queue, a request to each, to obtain a token from the queue, and continues.

Cluster limiting

For example, in order to limit the number of visits is a resource for each user or merchant, 5s only visit twice, or call one day only 1000, this demand, single restrictor can not be achieved, then we need to limit cluster stream implementation.

How to achieve? To control the number of visits, certainly we need a counter, and the counter can only be stored in third-party services, such as redis.

Probably thinking: every time there is related operations when it is sent to a server redis incr command, such as the number of times a user visited / index interface needs to be limited, only stitching user id and interface names key generation redis each time the user when accessing this interface, only need to perform incr command for this key, the expiration time to bring in this key, it can achieve access frequency specified time.

It can also be used to limit the flow of message queues, such as the processing flow of the high-actuation when clipping.

Service downgraded

It refers to downgrade certain system functions to reduce traffic or interface, may be a part of the functions only, or may be completely stopped all the functions.

Case

1. Dual 11 orders temporarily not allowed to modify the shipping address
2. Forum, downgraded to only see posts can not post messages
3. App log upload interface can be completely stopped for some time, this time APP can not upload log

the way

Common ways to achieve downgrade: Independent relegation system. It simply can be configured in the configuration center.

The downgrade operation to separate a single system can implement complex rights management, batch operation and other functions. The basic architecture is as follows:

Fuse

The purpose of the fuse is dependent on external circumstances to deal with system failures

Case

1. X function A service of B services depend on an interface, when the interface is slow response B service time, X features A service response certainly be slowed down, leading to further services are thread A card processing function on X, a case other functions and services will be jammed or respond very slowly
2. After addition of fusing mechanism, A Service B Service request not this interface, the internal A service request is found as long as the interfaces and services B immediately returns an error, so as to avoid slowing down or even throughout the service A slow death

achieve

1. The key is the need for a unified API call level, to statistical sampling or by the API call level, if the interface calls scattered throughout the code can not be a unified treatment
2. Another key design threshold value, for example 30% of the request response time exceeds 1 minute 1 second fuse, "1 minute" The strategy of "30%" "1 second" have an impact on the final results fuse
3. In practice generally the first threshold value is determined and then based on the analysis, and then observe the effect on the line, then tune

https://blog.csdn.net/bjpowernode_com/article/details/85162473