Microservice Protection

1. Getting to know Sentinel

1. Avalanche problem and solution

A service failure in the microservice call link ,

Causes all microservices in the entire link to fail

available , this is an avalanche

There are four common solutions:

① Timeout processing : set the timeout time, the request timeout

If there is no response after a certain period of time, an error message will be returned

don't wait forever

② Bulkhead mode : limited to the ones that can be used by each business

The number of threads to avoid exhausting the resources of the entire tomcat

source, so also called thread isolation

③Fusing downgrade : performed by the circuit breaker statistics business

The abnormal ratio of , if it exceeds the threshold, it will be blown

This business intercepts all requests to access this business

④Flow control : QPS that restricts business access,

Avoid service failure due to sudden increase in traffic

2. Comparison of service protection technologies

3. Sentinel introduction and installation

Sentinel is a microservice open sourced by Alibaba

flow control components

https://sentinelguard.io/zh-cn/index.html Sentinel

Has the following characteristics:

① Rich application scenarios

② Complete real-time monitoring

③ Extensive open source ecology

④ Perfect SPI extension point

(2) Install Sentinel Console:

① Prepare sentinel-dashboard-1.8.1.jar

② Execute java -jar sentinel-dashboard-1.8.1.jar

③ Visit: localhost:8080 to see the console page,

The default account and password are sentinel

(3) Modify Sentinel's default port, account, and password

java -jar sentinel-dashboard-1.8.1.jar -Dserver.port=8080

4. Microservice integration Sentinel

Integrate Sentinel in order-service and connect

Sentinel console, the steps are as follows:

(1) Introduce sentinel dependencies:

<!--sentinel-->
<dependency>
    <groupId>com.alibaba.cloud</groupId> 
    <artifactId>spring-cloud-starter-alibaba-sentinel</artifactId>
</dependency>

(2) Configure console address

Modify the application.yaml file and add the following content:

server:
  port: 8088
spring:
  cloud: 
    sentinel:
      transport:
        dashboard: localhost:8080

(3) Access any endpoint of the microservice to trigger sentinel monitoring

Visit http://localhost:8088/order/101

2. Current limiting rules

1. Quick Start

Cluster point link : it is the call link in the project, the link

Each interface being monitored in is a

resource

By default sentinel will monitor each of SpringMVC

An endpoint ( Endpoint ), so SpringMVC's

Each endpoint ( Endpoint ) is the call chain

a resource. Flow control, fusing, etc. are all aimed at the cluster point chain

resources on the way, so we can click

Correspond to the button behind the resource to set the rules

Click flow control behind resources /order/{orderId}

button , the form will pop up, and the form can be

Add flow control rules

Meaning in the picture: limit the resource /order/{orderId}

The stand-alone QPS is 1, that is, only 1 request is allowed per second .

Exceeding requests will be intercepted and an error will be reported

2. Flow control mode

When adding a current limiting rule, click the advanced option, you can choose

Choose from three flow control modes:

① Direct : Statistics of current resource requests, trigger threshold

When the value is set, the current resource is directly limited , which is also the default

mode

②Association : Statistics related to another resource related to the current resource

Resource, when the threshold is triggered, the current resource is limited

③Link : Statistics on the access to this resource from the specified link

request, when the threshold is triggered, the specified link is limited

flow

(2) Association

The association mode can be used if the following conditions are met:

① Two competing resources

② One with higher priority and one with lower priority

When the /write resource access triggers the threshold, the

/read resource current limit, avoid affecting /write resource

(3) link

For example, there are two request links:

/test1 → /common

/test2 → /common

If you only want statistics to enter /common from /test2

request, you can configure it like this

① Sentinel only marks the parties in the Controller by default.

method as a resource, if you want to mark other methods, you need

Using the @SentineResource annotation

@SentineResource("goods")
public void queryGoods() {
        System.err.println("查询商品")；
}

② Sentinel will make the Controller method by default

Context integration, resulting in loss of flow control in link mode

effect

You need to modify application.yml and add configuration:

spring：
  cloud：
    sentinel：
      web-context-unify：false # 关闭 context 整合

3. Flow control effect

The flow control effect means that when the request reaches the flow control threshold, the

the measures taken

Includes three types:

①Fast failure : After the threshold is reached, new requests will be rejected

Reject immediately and throw FlowException

(default)

② warm up : preheating mode , please

The request is also rejected and an exception is thrown, but this mode

The threshold will change dynamically , gradually

Added to the maximum threshold

③Wait in line : let all requests follow the order

Queued execution , the interval between two requests cannot be less than specified

Timing

4. Hotspot parameter current limit

The previous current limit is to count all requests to access a resource ,

Determine whether the QPS threshold is exceeded, and the current limit of the hotspot parameter

It is to count the requests with the same parameter value separately , and judge whether the

Passed the QPS threshold

(1) Configuration example:

Parameter 0 (the first parameter) of the hot resource

For statistics, the number of requests with the same parameter value per second cannot be

more than 5

(2) In the advanced options, you can set some parameters

Exception configuration:

Combined with the previous configuration, the meaning here is for number 0

The long type parameter current limit, the same parameter every 1 second

QPS cannot exceed 5

There are two exceptions:

① If the parameter value is 100, the allowed

QPS is 10

② If the parameter value is 101, the allowed

QPS is 15

Note: The hotspot parameter current limit is for the default SpringMVC

Invalid resource

3. Isolation and downgrade

Although current limiting can try to avoid problems caused by high concurrency

The service fails, but the service can also fail for other reasons

faults, but to control these faults within a certain range, to avoid

Avoiding avalanches depends on thread isolation (bulkwall mode) and melting

No downgrading means, whether it is thread isolation or fusing

Downgrading is the protection of the client (caller)

1. FeignClient integrates Sentinel

(1) Modify the application.yml file of OrderService,

Enable Feign's Sentinel function

feign:
  sentinel:
    enabled: true # 开启Feign的Sentinel功能

(2) Write downgrade logic for FeignClient after failure

Method 1: FallbackClass , unable to call remotely

exception handling

Method 2: FallbackFactory , you can call the remote

Use the exception to handle, we choose this

Write and register FallbackFactory for FeignClient

for Bean

@Slf4j
@Bean
public class UserClientFallbackFactory implements FallbackFactory<UserClient> {
    
    @Override
    public UserClient create(Throwable throwable) {
        // 创建UserClient接口实现类，实现其中的方法，编写失败降级的处理逻辑
        return new UserClient() {
            @Override
            public User findById(Long id) {
                // 记录异常信息
                log.error("查询用户失败", throwable);
                // 根据业务需求返回默认的数据，这里是空用户
                return new User();
            }
        };
    }
}

(3) In the UserClient interface in the feeding-api project

Use UserClientFallbackFactory

@FeignClient(value = "userservice", fallbackFactory = UserClientFallbackFactory.class)
public interface UserClient {

    @GetMapping("/user/{id}")
    User findById(@PathVariable("id") Long id);
}

2. Thread isolation (bulkwall mode)

Thread isolation can be achieved in two ways:

① Thread pool isolation

② Semaphore isolation (Sentinel uses it by default)

(1) Semaphore isolation

Advantages: lightweight, no additional overhead

Disadvantage: does not support active timeout

Does not support asynchronous calls

Scenario: high-frequency call, high fan-out

(2) Thread pool isolation

Advantages: support active timeout

Support for asynchronous calls

Disadvantages: the additional overhead of threads is relatively large

Scenario: low fan-out

Low fan-out : that is to say, a small or moderate amount of fan-out in a class

use other classes

High fanout (more than about 7): indicates that a class uses a large

other classes as it may become overly complex

When adding a throttling rule, you can choose two threshold types:

QPS: the number of requests per second

Number of threads: is the tomcat thread that this resource can use

The maximum value of the number, that is, by limiting the thread

Quantity , implementing bulkhead mode

3. Fuse degradation

Fuse downgrade is an important means to solve the avalanche problem, its

The idea is to use the circuit breaker to count the abnormal proportion of service calls,

Slow request ratio, if the threshold is exceeded, the server will be blown

service.

That is, intercept all requests to access the service; and when the service

When restored, the circuit breaker will allow access to the service

There are three types of circuit breaker fusing strategies:

① Slow call

② Abnormal ratio

③ Abnormal number

(1) Slow call

Requests whose service response time (RT) is longer than the specified time

Ask for statistics on the proportion of slow calls within a unit duration, exceeding

Fuse if the threshold is exceeded

A call with an RT exceeding 500ms is a slow call, and the latest statistics

For requests within 10000ms, if the number of requests exceeds 10,

And the proportion of slow calls is not less than 0.5, then the fuse will be triggered .

The off time is 5 seconds, then enter the half-open state, put

Run a request for testing

(2) Abnormal ratio

The proportion of abnormal calls within the statistical unit duration, exceeding

Threshold then melts

Count the requests within the last 1000ms, if the request volume

More than 10 times, and the abnormal ratio is not less than 0.5, then

Trigger the fuse, the duration of the fuse is 5 seconds, and then enter

Half-open state, release a request for testing

(3) Abnormal number

Statistics of the number of abnormal calls within the unit duration, exceeding

Threshold is blown

4. Authorization rules

(1) Authorization rules

Authorization rules can control the source of the caller,

There are two ways:

Whitelist: callers whose origin is in the whitelist

allow access

Blacklist: callers whose origin is in the blacklist

no access

For example, we restrict access to only requests from the gateway

orde-service, then fill in the gateway in the flow control application

The name

① Sentinel is through RequestOriginParser

The parseOrigin of this interface to get the requested

source of

public interface RequestOriginParser {
	/**
	* 从请求request对象中获取origin，获取方式自定义
	*/
	String parseOrigin(HttpServletRequest request);
}

In Sentinel, the parseOrigin method of this interface

The default is always returned , that is, regardless of the request

Whether the open source comes from the gateway or the browser, Sentinel does not

Can't distinguish between the two requests

Need to implement this interface , write its business logic, let

The request from the gateway is different from the request from the browser

the result of

② Get a request named origin from request

header, as the value of origin

@Component
public class HeaderOriginParser implements RequestOriginParser {
    @Override
    public String parseOrigin(HttpServletRequest httpServletRequest) {
        // 尝试获取请求头
        String origin = httpServletRequest.getHeader("origin");
        // 非空判断
        if (StringUtils.isEmpty(origin)){
            origin = "blank";
        }
        return origin;
    }
}

③ In the gateway service, use the global process of the gateway

The filter adds an origin header named gateway

spring:
  cloud:
    gateway:
      default-filters:
        - AddRequestHeader=origin,gateway # 添加名为origin的请求头，值为gateway

④ Configure authorization rules for /order/{orderId}:

(2) Custom abnormal results

By default, current limiting, downgrading, and authorization interception occur

, an exception is thrown to the caller

If you want to customize the return result when an exception occurs, you need to implement

BlockExceptionHandler interface:

public interface BlockExceptionHandler {
    // 处理请求被限流、降级、授权拦截时抛出的异常
    void handle(HttpServletRequest var1, HttpServletResponse var2, BlockException var3) throws Exception;
}

BlockException contains many subclasses to deal with different

scene:

abnormal	illustrate
FlowException	Current limit exception
ParamFlowException	Abnormal hotspot parameter current limit
DegradeException	downgrade exception
AuthorityException	Authorization rule exception
SystemBlockException	System rule exception

Define a class in order-service and implement the BlockExceptionHandler interface:

@Component
public class SentinelExceptionHandler implements BlockExceptionHandler {
    @Override
    public void handle(HttpServletRequest httpServletRequest, HttpServletResponse httpServletResponse, BlockException e) throws Exception {
        String msg = "未知异常";
        int status = 429;

        if (e instanceof FlowException){
            msg = "请求被限流了";
        } else if (e instanceof ParamFlowException){
            msg = "请求被热点参数限流了";
        } else if (e instanceof DegradeException){
            msg = "请求被降级了";
        } else if (e instanceof AuthorityException) {
            msg = "没有权限访问";
            status = 401;
        }

        httpServletResponse.setContentType("application/json;charset=utf-8");
        httpServletResponse.setStatus(status);
        httpServletResponse.getWriter().write("{\"msg\": " + msg + ", \"status\":" + status + "}");
    }
}

5. Persistence of rules

1. Rule management mode

Sentinel's console rule management has three modes:

① Original mode: The rules configured in the console are directly pushed to

Sentinel client, which is our application, and

Saved in memory, it will be lost when the service is restarted

② pull mode: the console pushes the configured rules to

Sentinel client, and the client saves the configuration rules

Stored in local files or databases , it will be regularly deleted in the future

Query and update local rules in local files or databases

③ push mode: the console pushes the configuration rules to the remote

Configuration center , such as Nacos or Zookeeper,

Sentinel client monitors Nocas to obtain configuration changes

push message to complete the local configuration update (recommended)

2. Implement push mode

The push mode is the most complex to implement because it relies on

nacos, and need to change the source code of Sentinel console,

The overall steps are as follows:

(1) Modify the order-service service to make it listen

Nacos Configuration Center

① Introduce dependency

<dependency>
    <groupId>com.alibaba.csp</groupId>
    <artifactId>sentinel-datasource-nacos</artifactId>
</dependency>

② Configure nacos address

spring:
  cloud:
    sentinel:
      datasource:
        flow:
          nacos:
            server-addr: localhost:8848 # nacos地址
            dataId: orderservice-flow-rules
            groupId: SENTINEL_GROUP
            rule-type: flow   # 还可以是degrade、authority、param-flow
        degrade:
          nacos:
              server-addr: localhost:8848 # nacos地址
              dataId: orderservice-degrade-rules
              groupId: SENTINEL_GROUP
              rule-type: degrade   # 还可以是degrade、authority、param-flow

(2) Modify Sentinel-dashboard source code, configuration

nacos data source

Modify the pom file of the source code, the sentinel-

Remove the scope that datasource-nacos depends on

<dependency>
    <groupId>com.alibaba.csp</groupId>
    <artifactId>sentinel-datasource-nacos</artifactId>
    <!--<scope>test</scope>-->
</dependency>

(3) Modify the source code of Sentinel-dashboard and modify the front end

page

① Copy the nacos code in the test directory to the main

com.alibaba.csp.sentinel.dashboard.rule 包

② Modify NacosConfig under the newly copied nacos package

Class, modify the nacos address in it

@Bean
public ConfigService nacosConfigService() throws Exception {
    return ConfigFactory.createConfigService("localhost:8848"); 
}

④ Modify com.alibaba.csp.sentinel.dashboard.

FlowControllerV2 class under controller.v2 package

(4) Recompile and package Sentinel-dashboard source code

① Modify src/main/webapp/resources/app/scripts/

The sidebar.html file in the directives/sidebar/ directory ,

Turn on this part of the comment

② Modify the text in it

<li ui-sref-active="active" ng-if="entry.appType==0">
    <a ui-sref="dashboard.flow({app: entry.app})">
        <i class="glyphicon glyphicon-filter"></i>&nbsp;&nbsp;流控规则-NACOS</a>
</li>

Microservice Protection

Guess you like