Avalanche problem : A service failure in the microservice call link causes all microservices in the entire link to be unavailable. This is an avalanche.

There are four common ways to solve avalanche problems:

Timeout processing: Set the timeout time. If the request exceeds a certain time and there is no response, an error message will be returned. There will be no endless waiting.

Bulkhead mode: Limit the number of threads that each business can use to avoid exhausting the entire tomcat resources, so it is also called thread isolation.

Circuit breaker degradation: The circuit breaker counts the abnormal proportion of business execution. If the threshold is exceeded, the business will be circuit breaker and all requests to access the business will be intercepted.

Traffic control: QPS that limits business access to avoid service failures due to sudden increases in traffic.

How to avoid service failures caused by instantaneous high concurrent traffic?

flow control

How to avoid avalanche problems caused by service failures?

Timeout processing thread isolation degradation circuit breaker

1.1Get to know Sentinel

Sentinel is a microservice flow control component open sourced by Alibaba. Official website address: https://sentinelguard.io/zh-cn/index.html Sentinel has the following characteristics:

Rich application scenarios : Sentinel has taken over the core scenarios of Alibaba’s Double Eleven traffic promotion in the past 10 years, such as flash sales (that is, burst traffic is controlled within the range that the system capacity can bear), message peak-shaving and valley-filling, and cluster traffic control. , real-time fusing of downstream unavailable applications, etc.

Complete real-time monitoring : Sentinel also provides real-time monitoring functions. You can see the second-level data of a single machine connected to the application in the console, and even the summary operation status of a cluster of less than 500 machines.

Extensive open source ecosystem : Sentinel provides out-of-the-box integration modules with other open source frameworks/libraries, such as integration with Spring Cloud, Dubbo, and gRPC. You only need to introduce the corresponding dependencies and perform simple configuration to quickly connect to Sentinel.

Complete SPI extension point : Sentinel provides an easy-to-use, complete SPI extension interface. You can quickly customize logic by implementing extension interfaces. For example, customized rule management, adapting dynamic data sources, etc.

1.1.1Install Sentinel console

Sentinel officially provides a UI console to facilitate current limiting settings on the system. You can download it on GitHub.

Then visit: localhost:8080 to see the console page. The default account and password are sentinel.

illustrate:

1.2 Microservice integration with Sentinel

(1) The parent project introduces sentinel dependency

<!--sentinel-->
        <dependency>
            <groupId>com.alibaba.cloud</groupId>
            <artifactId>spring-cloud-starter-alibaba-sentinel</artifactId>
        </dependency>

(2) Configure the console address

#sentinel控制台的地址
spring.cloud.sentinel.transport.dashboard=localhost:8090

(3) Access interface

1.3 Cluster point link

Cluster point link: It is the calling link within the project. Each interface monitored in the link is a resource. By default, sentinel will monitor each endpoint (Endpoint) of SpringMVC, so each endpoint (Endpoint) of SpringMVC is a resource in the call link. Flow control, circuit breaker, etc. are all set for the resources in the cluster point link, so we can click the button behind the corresponding resource to set the rules:

Requirement: Set flow control rules for the resource /product/getName. QPS cannot exceed 5. Then use jemeter to test.

2 flow control mode

When adding a current limiting rule, click Advanced Options and you can choose from three flow control modes: Direct: counts requests for the current resource, and directly limits the current resource when the threshold is triggered. This is also the default mode. Association: counts requests for other resources related to the current resource. For a resource, when the threshold is triggered, the current resource is restricted. Link: Statistics of requests to access this resource from the specified link are counted. When the threshold is triggered, the specified link is restricted.

2.1 Flow control mode-association

Association mode: Counts another resource related to the current resource. When the threshold is triggered, the current resource is limited.

Usage scenario: For example, the user needs to modify the order status when paying, and at the same time the user needs to query the order. Query and modification operations will compete for database locks and cause competition. The business requirement is limited payment and order update business, so when the order business trigger threshold is modified, the query order business flow needs to be limited.

After stress testing, it was found that when the write exceeds the threshold, the read flow is limited.

Association mode can be used if the following conditions are met:

Two competing resources, one with higher priority and one with lower priority

2.3 Flow control mode-link

Link mode: Only make statistics on requests to access this resource from the specified link to determine whether the threshold is exceeded.

It was found that productCommon was not restricted:

Sentinel only marks methods in the Controller as resources by default. If you want to mark other methods, you need to use the @SentinelResource annotation.

Sentinel will integrate the Controller method into the context by default, causing the flow control of the link mode to fail. You need to modify application.yml and add

After stress testing, it was found that requests initiated from /product/query have traffic restrictions, but requests initiated from /product/save do not.

3 Flow control effect

Flow control effect refers to the measures that should be taken when the request reaches the flow control threshold, including three types:

Fail fast: After reaching the threshold, new requests will be rejected immediately and a FlowException will be thrown. This is the default processing method.

warm up: warm-up mode, requests that exceed the threshold are also rejected and exceptions are thrown. But this mode threshold changes dynamically, gradually increasing from a smaller value to the maximum threshold.

Queuing and waiting: Queue all requests for execution in order. The interval between two requests cannot be less than the specified time.

3.1 Flow control effect-warm up

Warm up, also called warm-up mode, is a solution to cold start of services. The initial value of the request threshold is threshold/coldFactor. After a specified period of time, it will gradually increase to the threshold value. The default value of coldFactor is 3. For example, if I set the threshold of QPS to 10 and the warm-up time to 5 seconds, then the initial threshold is 10 / 3, which is 3, and then gradually increases to 10 after 5 seconds.

After stress testing, it was found that the request threshold will slowly increase until it reaches the maximum threshold.

3.2 Flow control effect - waiting in line

When requests exceed the QPS threshold, fast fail and warm up will reject new requests and throw an exception. Queuing and waiting is to put all requests into a queue and then execute them in sequence according to the time interval allowed by the threshold. Subsequent requests must wait for the previous execution to complete. If the expected waiting time of the request exceeds the maximum duration, it will be rejected. At one moment, the service is busy, and at other moments, the service is idle. For example: QPS = 5, which means that a request in the queue is processed every 200ms; timeout = 2000, which means that requests that are expected to wait for more than 2000ms will be rejected and an exception will be thrown.

After stress testing, all requests were successful at first, but gradually some were not successful because the timeout period was exceeded, causing the request to be disallowed. In the end, all requests were successful because the previous requests had been processed, so subsequent requests could be successful.

What are the flow control effects?

Fast failure: When QPS exceeds the threshold, new requests are rejected

Warm up: When the QPS exceeds the threshold, new requests are rejected; the QPS threshold is gradually increased to avoid service downtime caused by high concurrency during cold start.

Waiting in queue: The request will enter the queue and be executed sequentially according to the time interval allowed by the threshold; if the expected waiting time of the request is longer than the timeout, it will be rejected directly.

4 Hotspot parameters current limit

The previous current limit was to count all requests to access a certain resource to determine whether it exceeded the QPS threshold. The hotspot parameter current limiting is to separately count requests with the same parameter value to determine whether the QPS threshold is exceeded.

In the advanced options of hotspot parameter current limiting, you can set exception configurations for some parameters:

Combined with the previous configuration, the meaning here is to limit the current flow of the int type parameter No. 0. The QPS of the same parameter every 1 second cannot exceed 2.

There are two exceptions: If the parameter value is 101, the allowed QPS per 1 second is 4. If the parameter value is 102, the allowed QPS per 1 second is 5

Note: Hotspot parameter current limiting is not valid for default SpringMVC resources! ! ! ! ! ! !

5 Quarantine and downgrade

5.1 Quarantine and downgrade

Although current limiting can try to avoid service failures caused by high concurrency, services can also fail for other reasons. To control these faults within a certain range and avoid avalanches, we must rely on thread isolation (bulkhead mode) and circuit breaker degradation. Whether it is thread isolation or circuit breaker downgrade, they all protect the client (caller).

5.2 Feign integrates Sentinel

In SpringCloud, microservice calls are implemented through Feign, so client protection must integrate Feign and Sentinel.

1. Modify the application.yml file of OrderService, enable Feign’s Sentinel function, and don’t forget to add the sentinel dependency.

2. Write downgrade logic for FeignClient after failure

Method 1: FallbackClass, unable to handle exceptions in remote calls

Method 2: FallbackFactory, which can handle exceptions in remote calls, we choose this

Write your own fallBackFactory class and implement the FallbackFactory interface
@Component
public class MyFallBackFactory implements FallbackFactory<OpenFeign> {
    @Override
    public OpenFeign create(Throwable throwable) {
        return new OpenFeign() {
            @Override
            public Product getProduct(Integer id) {
                Product product = new Product();
                product.setPid(-1L);
                product.setPname("服务器出现故障,请稍后再试");
                return product;
            }
        };
    }
}
ModifyOpenFeign

Avalanche solution supported by Sentinel: Thread isolation (silo wall mode) Downgrade circuit breaker

Steps for Feign to integrate Sentinel:

Configure in application.yml: feign.sentienl.enable=true

Write FallbackFactory for FeignClient and register it as a Bean

Configure FallbackFactory to FeignClient

6 thread isolation

There are two ways to implement thread isolation: thread pool isolation and semaphore isolation (used by Sentinel by default)

6.1 Thread isolation (bulkhead mode)

When adding a current limiting rule, you can choose from two threshold types:

QPS: It is the number of requests per second, which has been demonstrated in the quick start

Number of threads: It is the maximum number of tomcat threads that can be used by this resource. That is, by limiting the number of threads, the bulkhead mode is achieved.

Example:

7 circuit breaker downgrade

7.1 Circuit breaker strategy-slow call

Circuit breaker degradation is an important means to solve the avalanche problem. The idea is that the circuit breaker counts the abnormal proportion and slow request proportion of service calls. If the threshold is exceeded, the service will be cut off. That is, all requests to access the service are intercepted; and when the service is restored, the circuit breaker will release requests to access the service.

Written in the control layer in product

7.2 Circuit breaker strategy - abnormal proportion and number of exceptions

There are three circuit breaker fusing strategies: slow calls, exception ratio or number of exceptions. Exception ratio or number of exceptions: counts calls within a specified time. If the number of calls exceeds the specified number of requests, and the proportion of exceptions reaches the set ratio threshold (or exceeds Specify the number of exceptions), the circuit breaker is triggered. For example:

8 Custom exception results

We define a class in order-service and implement the BlockExceptionHandler interface:

@Component
public class SentinelBlockHandler implements BlockExceptionHandler {
    @Override
    public void handle(HttpServletRequest httpServletRequest, HttpServletResponse httpServletResponse, BlockException e) throws Exception {
        String msg = "未知异常";
        if (e instanceof FlowException) {
            msg = "请求被限流了！";
        } else if (e instanceof DegradeException) {
            msg = "请求被降级了！";
        } else if (e instanceof ParamFlowException) {
            msg = "热点参数限流！";
        } else if (e instanceof AuthorityException) {
            msg = "请求没有权限！";
        }
        httpServletResponse.setContentType("application/json;charset=utf-8");
        httpServletResponse.getWriter().print("{\"message\": \"" + msg + "\"}");
    }
}

After setting the flow control casually, you can see that the abnormal results defined by yourself are used.

SpringCloud Microservice-----Microservice Protection Sentinel

1. First introduction to Sentinel