Spring Cloud (3) Hystrix circuit breaker service downgrade, service fusing, service monitoring case details

Seven, Hystrix circuit breaker

7.1 Introduction

• Problems Facing Distributed Systems
  Applications in complex distributed architectures have dozens of dependencies, each of which will inevitably fail at some point.

  When multiple microservices are called, suppose microservice A calls microservice B and microservice C, and microservice B and microservice C call other microservices. This is the so-called "fan-out". If the call response time of a microservice on the fan-out link is too long or unavailable, the call to microservice A will occupy more and more system resources, causing the system to crash, the so-called "avalanche effect".

  For high-traffic applications, a single backend dependency can saturate all resources on all servers within seconds. Worse than failing, these applications can also cause increased latency between services, straining backup queues, threads, and other system resources, leading to more cascading failures throughout the system. These all represent the need to isolate and manage failures and latencies so that the failure of a single dependency cannot bring down an entire application or system.
  Therefore, usually when you find that an instance under a module fails, the module will still receive traffic at this time, and then the problematic module calls other modules, which will cause cascading failures, or avalanches. .

• Hystrix

In a distributed system, many dependencies will inevitably fail to call, such as timeouts, exceptions, etc. Hystrix can ensure that in the case of a dependency problem, it will not cause the overall service to fail, avoid cascading failures, and improve distributed systems. flexibility.

The "circuit breaker" itself is a switching device. When a service unit fails, through the fault monitoring of the circuit breaker (similar to a blown fuse), an expected and processable alternative response (FallBack) is returned to the caller. , instead of waiting for a long time or throwing an exception that the caller cannot handle, this ensures that the thread of the service caller will not be occupied for a long time and unnecessarily, thereby avoiding the spread of faults in the distributed system, and even avalanche.

• Functions of Hystrix

服务降级

Don’t let the client wait and return a friendly prompt immediately (abnormal program operation, timeout, service fuse triggers service degradation, thread pool/semaphore full will also cause service degradation)

服务熔断

After reaching the maximum service access, directly deny the access, then call the service downgrade method and return a friendly prompt

服务限流

Seckill high concurrency and other operations, it is strictly forbidden to swarm over and crowd, everyone queues up, N per second, and proceed in an orderly manner

接近实时的监控

• Hystrix discontinued maintenance

Official website:https://github.com/Netflix/Hystrix/wiki/How-To-Use

Maintenance stop instructions:https://github.com/Netflix/Hystrix

7.2 Detailed explanation of Hystrix service downgrade case

新建cloud-provider-hystrix-payment8001

7.2.1 pom file

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <artifactId>springcloud</artifactId>
        <groupId>com.yyds</groupId>
        <version>1.0-SNAPSHOT</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>

    <artifactId>cloud-provider-hystrix-payment8001</artifactId>

    <dependencies>

        <!--hystrix-->
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-hystrix</artifactId>
        </dependency>

        <!--eureka-client-->
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>


        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>


        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-devtools</artifactId>
            <scope>runtime</scope>
            <optional>true</optional>
        </dependency>

        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
        </dependency>
    </dependencies>

</project>

7.2.2 yml file

server:
  port: 8001


spring:
  application:
    name: cloud-provider-hystrix-payment


eureka:
  client:
    service-url:
      defaultZone: http://localhost:7001/eureka
#      defaultZone: http://eureka7001.com:7001/eureka,http://eureka7002.com:7002/eureka  # 集群版本
    register-with-eureka: true
    fetch-registry: true
  instance:
    instance-id: payment8001
    prefer-ip-address: true  # 显示ip

7.2.3 Main startup class

package com.yyds.springcloud;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.netflix.eureka.EnableEurekaClient;

@SpringBootApplication
@EnableEurekaClient
public class HystrixPaymentMain8001 {
    
    

    public static void main(String[] args) {
    
    
        SpringApplication.run(HystrixPaymentMain8001.class, args);
        System.out.println("*******************HystrixPaymentMain8001启动成功********************");
    }
}

7.2.4 Business class

package com.yyds.springcloud.controller;

import com.yyds.springcloud.service.PaymentService;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RestController;

import javax.annotation.Resource;

@RestController
@Slf4j
public class PaymentController {
    
    

    @Resource
    private PaymentService paymentService;

    @Value("${server.port}")
    private String serverPort;


    @GetMapping("/payment/hystrix/ok/{id}")
    public String paymentInfo_OK(@PathVariable("id") Integer id) {
    
    
        String result = paymentService.paymentInfo_OK(id);
        log.info("****result: " + result);
        return result;
    }

    @GetMapping("/payment/hystrix/timeout/{id}")
    public String paymentInfo_TimeOut(@PathVariable("id") Integer id) throws InterruptedException {
    
    
        String result = paymentService.paymentInfo_TimeOut(id);
        log.info("****result: " + result);
        return result;
    }

}

package com.yyds.springcloud.service;

import org.springframework.stereotype.Service;
import java.util.concurrent.TimeUnit;

@Service
public class PaymentService {
    
    

    /**
     * 正常访问，一切OK
     *      
     */
    public String paymentInfo_OK(Integer id) {
    
    
        return "线程池:" + Thread.currentThread().getName() + "paymentInfo_OK,id: " + id + "\t" + "O(∩_∩)O";
    }

    /**
     * 超时访问，演示降级
     *      
     */
    public String paymentInfo_TimeOut(Integer id) {
    
    
        try {
    
    
            TimeUnit.SECONDS.sleep(3);
        } catch (InterruptedException e) {
    
    
            e.printStackTrace();
        }
        return "线程池:" + Thread.currentThread().getName() + "paymentInfo_TimeOut,id: " + id + "\t" + "O(∩_∩)O，耗费3秒";
    }

}

7.2.5 Testing

Test interface:http://localhost:8001/payment/hystrix/ok/1

Test interface:http://localhost:8001/payment/hystrix/timeout/1

The above test, under normal circumstances, is no problem. But in the case of high concurrency, for example: open Jmeter, 20,000 concurrency crushes 8001, 20,000 requests go to the paymentInfo_TimeOut service, and both interfaces are turning around by themselves. This is because tomcat's default number of worker threads is fully loaded, and there are no extra threads to decompose pressure and processing.

If the external consumer 80 also comes to visit at this time, the consumer can only wait, and eventually the consumer 80 is dissatisfied, and the server 8001 is directly dragged to death.

therefore,对于超时导致服务器变慢(转圈),超时不再等待；对于出错(宕机或程序运行出错)，出错要有兜底。

Specifically:

• The other party's service (8001) has timed out, and the caller (80) cannot wait forever, there must be a service downgrade

• The other party's service (8001) is down, and the caller (80) cannot wait forever, there must be a service downgrade

• The other party's service (8001) is OK, the caller (80) has a failure or has its own requirements (its own waiting time is less than the service provider), and it handles the downgrade by itself

7.2.6 Service downgrade configuration @HystrixCommand

cloud-provider-hystrix-payment8001 sets the peak value of its own call timeout, and it can run normally within the peak value. If it exceeds the peak value, it needs to be dealt with in a comprehensive way, and it will be used as a service downgrade fallback.

业务类@HystrixCommand

package com.yyds.springcloud.service;

import com.netflix.hystrix.contrib.javanica.annotation.HystrixCommand;
import com.netflix.hystrix.contrib.javanica.annotation.HystrixProperty;
import org.springframework.stereotype.Service;

import java.util.concurrent.TimeUnit;

@Service
public class PaymentService {
    
    
    /**
     * 正常访问，一切OK
     *      
     */
    public String paymentInfo_OK(Integer id) {
    
    
        return "线程池:" + Thread.currentThread().getName() + "paymentInfo_OK,id: " + id + "\t" + "O(∩_∩)O";
    }

    @HystrixCommand(
            fallbackMethod ="paymentInfo_TimeOutHandler",
            commandProperties = {
    
    
                @HystrixProperty(name="execution.isolation.thread.timeoutInMilliseconds",value="3000")
            }
    )
    public String paymentInfo_TimeOut(Integer id) {
    
    
            int second = 5;
            try {
    
    
                TimeUnit.SECONDS.sleep(second);
            } catch (InterruptedException e) {
    
    
                e.printStackTrace();
            }
            return "线程池:"+Thread.currentThread().getName()+"paymentInfo_TimeOut,id: "+id+"\t"+"O(∩_∩)O，耗费秒: "+second;
    }

   
    public String paymentInfo_TimeOutHandler(Integer id){
    
    
        return "/(ㄒoㄒ)/调用支付接口超时或异常：\t"+ "\t当前线程池名字" + Thread.currentThread().getName();
    }

}

Add annotation @EnableCircuitBreaker to the main startup class

package com.yyds.springcloud;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.client.circuitbreaker.EnableCircuitBreaker;
import org.springframework.cloud.netflix.eureka.EnableEurekaClient;

@SpringBootApplication
@EnableEurekaClient
@EnableCircuitBreaker // 添加此注解
public class HystrixPaymentMain8001 {
    
    

    public static void main(String[] args) {
    
    
        SpringApplication.run(HystrixPaymentMain8001.class, args);
        System.out.println("*******************HystrixPaymentMain8001启动成功********************");
    }

}

此时访问：http://localhost:8001/payment/hystrix/timeout/1

/(ㄒoㄒ)/调用支付接口超时或异常： 当前线程池名字HystrixTimer-1

7.2.7 consumer cloud-consumer-feign-hystrix-order80

new modulecloud-consumer-feign-hystrix-order80

pom文件

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <artifactId>springcloud</artifactId>
        <groupId>com.yyds</groupId>
        <version>1.0-SNAPSHOT</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>

    <artifactId>cloud-consumer-feign-hystrix-order80</artifactId>


    <dependencies>
        <!--openfeign-->
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-openfeign</artifactId>
        </dependency>
        <!--hystrix-->
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-hystrix</artifactId>
        </dependency>
        <!--eureka client-->
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
        </dependency>
       
        <!--web-->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
        <!--一般基础通用配置-->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-devtools</artifactId>
            <scope>runtime</scope>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>


</project>

yml文件

spring:
  application:
    name: cloud-provider-hystrix-order

# 添加下面配置信息
eureka:
  client:
    service-url:
      defaultZone: http://localhost:7001/eureka
    #      defaultZone: http://eureka7001.com:7001/eureka,http://eureka7002.com:7002/eureka  # 集群版本
    register-with-eureka: true # 是否从EurekaServer抓取已有的注册信息，默认为true。单节点无所谓，集群必须设置为true才能配合ribbon使用负载均衡
    fetch-registry: true
    
feign:
  hystrix:
    enabled: true

主启动类

package com.yyds.springcloud;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.netflix.hystrix.EnableHystrix;
import org.springframework.cloud.openfeign.EnableFeignClients;

@SpringBootApplication
@EnableFeignClients
@EnableHystrix // 添加此注解
public class OrderHystrixMain80
{
    
    
    public static void main(String[] args)
    {
    
    
        SpringApplication.run(OrderHystrixMain80.class,args);
    }
}

业务类

package com.yyds.springcloud.service;

import org.springframework.cloud.openfeign.FeignClient;
import org.springframework.stereotype.Component;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;

@Component
@FeignClient(value = "CLOUD-PROVIDER-HYSTRIX-PAYMENT")
public interface PaymentHystrixService {
    
    
    @GetMapping("/payment/hystrix/ok/{id}")
    String paymentInfo_OK(@PathVariable("id") Integer id);

    @GetMapping("/payment/hystrix/timeout/{id}")
    String paymentInfo_TimeOut(@PathVariable("id") Integer id);
}

package com.yyds.springcloud.controller;


import com.netflix.hystrix.contrib.javanica.annotation.HystrixCommand;
import com.netflix.hystrix.contrib.javanica.annotation.HystrixProperty;
import com.yyds.springcloud.service.PaymentHystrixService;
import lombok.extern.slf4j.Slf4j;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RestController;

import javax.annotation.Resource;

@RestController
@Slf4j
public class OrderHystirxController {
    
    

    @Resource
    private PaymentHystrixService paymentHystrixService;

    @GetMapping("/consumer/payment/hystrix/ok/{id}")
    public String paymentInfo_OK(@PathVariable("id") Integer id) {
    
    
        String result = paymentHystrixService.paymentInfo_OK(id);
        return result;
    }

    @GetMapping("/consumer/payment/hystrix/timeout/{id}")
    @HystrixCommand(fallbackMethod = "paymentTimeOutFallbackMethod",commandProperties = {
    
    
                   @HystrixProperty(name="execution.isolation.thread.timeoutInMilliseconds",value="1500")
            })
    public String paymentInfo_TimeOut(@PathVariable("id") Integer id)
    {
    
    
        String result = paymentHystrixService.paymentInfo_TimeOut(id);
        return result;
    }
    public String paymentTimeOutFallbackMethod(@PathVariable("id") Integer id) {
    
    
        return "我是消费者80,对方支付系统繁忙请10秒钟后再试或者自己运行出错请检查自己,o(╥﹏╥)o";
    }
}

测试

http://localhost/consumer/payment/hystrix/timeout/1

我是消费者80,对方支付系统繁忙请10秒钟后再试或者自己运行出错请检查自己,o(╥﹏╥)o

存在的问题

• Each business method corresponds to a bottom-up method, and the code expands

• Uniform and custom needs to be separated

7.2.8 Solve the problem__code bloat @DefaultProperties(defaultFallback="")

Modify business class

package com.yyds.springcloud.controller;

import com.netflix.hystrix.contrib.javanica.annotation.HystrixCommand;
import com.yyds.springcloud.service.PaymentHystrixService;
import com.netflix.hystrix.contrib.javanica.annotation.DefaultProperties;
import lombok.extern.slf4j.Slf4j;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RestController;

import javax.annotation.Resource;


@RestController
@Slf4j
@DefaultProperties(defaultFallback = "payment_Global_FallbackMethod")
public class OrderHystirxController
{
    
    
    @Resource
    private PaymentHystrixService paymentHystrixService;

    @GetMapping("/consumer/payment/hystrix/ok/{id}")
    public String paymentInfo_OK(@PathVariable("id") Integer id)
    {
    
    
        String result = paymentHystrixService.paymentInfo_OK(id);
        return result;
    }

    @GetMapping("/consumer/payment/hystrix/timeout/{id}")
    @HystrixCommand //加了@DefaultProperties属性注解，并且没有写具体方法名字，就用统一全局的
    public String paymentInfo_TimeOut(@PathVariable("id") Integer id)
    {
    
    
        int age = 10/0;
        String result = paymentHystrixService.paymentInfo_TimeOut(id);
        return result;
    }
    public String paymentTimeOutFallbackMethod(@PathVariable("id") Integer id)
    {
    
    
        return "我是消费者80,对方支付系统繁忙请10秒钟后再试或者自己运行出错请检查自己,o(╥﹏╥)o";
    }

    // 下面是全局fallback方法
    public String payment_Global_FallbackMethod()
    {
    
    
        return "Global异常处理信息，请稍后再试，/(ㄒoㄒ)/~~";
    }
}

test

http://localhost/consumer/payment/hystrix/timeout/1

Global异常处理信息，请稍后再试，/(ㄒoㄒ)/~~

7.2.9 Solve the problem__ mixed with business logic

Modify cloud-consumer-feign-hystrix-order80

package com.yyds.springcloud.service;


import org.springframework.cloud.openfeign.FeignClient;
import org.springframework.stereotype.Component;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;

@Component
@FeignClient(value = "CLOUD-PROVIDER-HYSTRIX-PAYMENT", fallback = PaymentFallbackService.class)
public interface PaymentHystrixService {
    
    

    @GetMapping("/payment/hystrix/ok/{id}")
    String paymentInfo_OK(@PathVariable("id") Integer id);

    @GetMapping("/payment/hystrix/timeout/{id}")
    String paymentInfo_TimeOut(@PathVariable("id") Integer id);

}

package com.yyds.springcloud.service;

import org.springframework.stereotype.Component;

@Component
public class PaymentFallbackService implements PaymentHystrixService {
    
    

    @Override
    public String paymentInfo_OK(Integer id) {
    
    
        return "paymentInfo_OK fall_back";
    }

    @Override
    public String paymentInfo_TimeOut(Integer id) {
    
    
        return "paymentInfo_TimeOut fall_back";
    }
}

test

http://localhost/consumer/payment/hystrix/ok/1
线程池:http-nio-8001-exec-2paymentInfo_OK,id: 1 O(∩_∩)O

# 此时关闭8001服务提供者，模拟宕机
http://localhost/consumer/payment/hystrix/ok/1
paymentInfo_OK fall_back

7.3 Detailed explanation of Hystrix service fuse case

熔断机制概述
The fuse mechanism is a microservice link protection mechanism to deal with the avalanche effect. When a microservice of the fanout link fails or the response time is too long, the service will be degraded, and then the call of the microservice of the node will be broken, and the wrong response information will be returned quickly. When it is detected that the microservice call response of the node is normal, the call link is resumed.

​ In the Spring Cloud framework, the fuse mechanism is implemented through Hystrix. Hystrix will monitor the status of calls between microservices. When the failed calls reach a certain threshold, the default is 20 call failures within 5 seconds, and the fuse mechanism will be activated. The annotation of the fuse mechanism is @HystrixCommand.

修改cloud-provider-hystrix-payment8001

Add this piece of code to PaymentService

  //=====================================服务熔断
    @HystrixCommand(fallbackMethod = "paymentCircuitBreaker_fallback",commandProperties = {
    
    
            @HystrixProperty(name = "circuitBreaker.enabled",value = "true"),// 是否开启断路器
            @HystrixProperty(name = "circuitBreaker.requestVolumeThreshold",value = "10"),// 请求次数
            @HystrixProperty(name = "circuitBreaker.sleepWindowInMilliseconds",value = "10000"), // 时间窗口期
            @HystrixProperty(name = "circuitBreaker.errorThresholdPercentage",value = "60"),// 失败率达到多少后跳闸
    })
    public String paymentCircuitBreaker(@PathVariable("id") Integer id)
    {
    
    
        if(id < 0)
        {
    
    
            throw new RuntimeException("******id 不能负数");
        }
        String serialNumber = IdUtil.simpleUUID();

        return Thread.currentThread().getName()+"\t"+"调用成功，流水号: " + serialNumber;
    }
    public String paymentCircuitBreaker_fallback(@PathVariable("id") Integer id)
    {
    
    
        return "id 不能负数，请稍后再试，/(ㄒoㄒ)/~~   id: " +id;
    }

Add in PaymentController

 @GetMapping("/payment/circuit/{id}")
    public String paymentCircuitBreaker(@PathVariable("id") Integer id) {
    
    
        String result = paymentService.paymentCircuitBreaker(id);
        log.info("****result: "+result);
        return result;
    }

test

持续点击 http://localhost:8001/payment/circuit/-1
# 出现下面结果
id 不能负数，请稍后再试，/(ㄒoㄒ)/~~ id: -1

# 此时访问下面地址，也不能访问
http://localhost:8001/payment/circuit/1
id 不能负数，请稍后再试，/(ㄒoㄒ)/~~ id: -1


# 等一段时间，能正常访问
http://localhost:8001/payment/circuit/1
hystrix-PaymentService-10 调用成功，流水号: b1d538146c1647019fb4620bc12e648e

7.4 Hystrix workflow

1. Create a HystrixCommand (used when the dependent service returns a single operation result) or HystrixObserableCommand (used when the dependent service returns multiple operation results) object.

2. Command execution. Among them, HystrixCommand implements the first two execution methods below; HystrixObservableCommand implements the latter two execution methods: execute(): synchronous execution, returning a single result object from the dependent service, or throwing an exception when an error occurs.

queue(): Asynchronous execution, directly returns a Future object, which contains a single result object to be returned when the service execution ends.

observe(): returns an Observable object, which represents multiple results of the operation, it is a Hot Observable (regardless of whether the "event source" has a "subscriber", the event will be published after creation, so for each Hot Observable A "subscriber" may start halfway from the "event source" and may only see part of the entire operation).

toObservable(): It also returns an Observable object, which also represents multiple results of the operation, but it returns a Cold Observable (events are not published when there is no "subscriber", but wait until there is a "subscriber" subscriber" before publishing events, so for Cold Observable subscribers, it is guaranteed to see the entire process of the entire operation from the beginning).

3. If the request cache function of the current command is enabled and the command cache hits, the cached result will be returned immediately in the form of an Observable object.

4. Check whether the circuit breaker is on. If the circuit breaker is open, then Hystrix will not execute the command, but transfer to the fallback processing logic (step 8); if the circuit breaker is closed, check whether there are available resources to execute the command (step 5).

5. Whether the thread pool/request queue/semaphore is full. If the command depends on the dedicated thread pool and request queue of the service, or the semaphore (when not using the thread pool) is already full, then Hystrix will not execute the command, but will transfer to the fallback processing logic (step 8) .

6. Hystrix will decide what method to use to request dependent services based on the method we wrote. HystrixCommand.run() : Returns a single result, or throws an exception. HystrixObservableCommand.construct(): Return an Observable object to emit multiple results, or send error notifications through onError.

7. Hystrix will report "success", "failure", "rejection", "timeout" and other information to the circuit breaker, and the circuit breaker will maintain a set of counters to count these data. The circuit breaker will use these statistics to decide whether to open the circuit breaker to "fuse/short circuit" the request of a dependent service.

8. When the command execution fails, Hystrix will enter fallback to try to roll back. We usually call this operation "service downgrade". The conditions that can cause service degradation processing are as follows: Step 4: The current command is in the "fuse/short circuit" state, and the circuit breaker is open. Step 5: When the thread pool, request queue or semaphore of the current command is full. Step 6: When HystrixObservableCommand.construct() or HystrixCommand.run() throws an exception.

9. When the Hystrix command is successfully executed, it will return the processing result directly or in the form of Observable.
Tips: If we don't implement the downgrade logic for the command or throw an exception in the downgrade processing logic, Hystrix will still return an Observable object, but it will not emit any result data, but notify the command to interrupt the request immediately through the onError method, and The exception that caused the command to fail is sent to the caller via the onError() method.

7.5 service monitoring hystrixDashboard

7.5.1 Detailed explanation of the case

In addition to isolating the calls of dependent services, Hystrix also provides that 准实时的调用监控（Hystrix Dashboard）Hystrix will continuously record the execution information of all requests initiated through Hystrix, and display them to users in the form of statistical reports and graphics, including how many requests are executed per second, how many are successful, and how many fail etc.

Netflix has implemented the monitoring of the above indicators through the hystrix-metrics-event-stream project. Spring Cloud also provides the integration of Hystrix Dashboard, which converts monitoring content into a visual interface.

New cloud-consumer-hystrix-dashboard9001

pom文件

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <artifactId>springcloud</artifactId>
        <groupId>com.yyds</groupId>
        <version>1.0-SNAPSHOT</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>

    <artifactId>cloud-consumer-hystrix-dashboard9001</artifactId>

    <dependencies>
        
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-hystrix-dashboard</artifactId>
        </dependency>
        
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-devtools</artifactId>
            <scope>runtime</scope>
            <optional>true</optional>
        </dependency>
        
        
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
        
    </dependencies>


</project>

yml文件

server:
  port: 9001

主启动类

package com.yyds.springcloud;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.netflix.hystrix.dashboard.EnableHystrixDashboard;


@SpringBootApplication
@EnableHystrixDashboard  // 添加此注解
public class HystrixDashboardMain9001
{
    
    
    public static void main(String[] args) {
    
    
            SpringApplication.run(HystrixDashboardMain9001.class, args);
    }
}

access:http://localhost:9001/hystrix

使用cloud-consumer-hystrix-dashboard9001监控cloud-provider-hystrix-payment8001

Modify cloud-provider-hystrix-payment8001

package com.yyds.springcloud;

import com.netflix.hystrix.contrib.metrics.eventstream.HystrixMetricsStreamServlet;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.web.servlet.ServletRegistrationBean;
import org.springframework.cloud.client.circuitbreaker.EnableCircuitBreaker;
import org.springframework.cloud.netflix.eureka.EnableEurekaClient;
import org.springframework.context.annotation.Bean;

@SpringBootApplication
@EnableEurekaClient
@EnableCircuitBreaker
public class HystrixPaymentMain8001 {
    
    

    public static void main(String[] args) {
    
    
        SpringApplication.run(HystrixPaymentMain8001.class, args);
        System.out.println("*******************HystrixPaymentMain8001启动成功********************");
    }

    /**
     *此配置是为了服务监控而配置，与服务容错本身无关，springcloud升级后的坑
     *ServletRegistrationBean因为springboot的默认路径不是"/hystrix.stream"，
     *只要在自己的项目里配置上下面的servlet就可以了
     */
    @Bean
    public ServletRegistrationBean getServlet() {
    
    
        HystrixMetricsStreamServlet streamServlet = new HystrixMetricsStreamServlet();
        ServletRegistrationBean registrationBean = new ServletRegistrationBean(streamServlet);
        registrationBean.setLoadOnStartup(1);
        registrationBean.addUrlMappings("/hystrix.stream");
        registrationBean.setName("HystrixMetricsStreamServlet");
        return registrationBean;
    }

}

Fill in the monitoring address

http://localhost:8001/hystrix.stream

test

不断访问http://localhost:8001/payment/circuit/1

不断访问http://localhost:8001/payment/circuit/-1

7.5.2 Hystrix Dashboard parameter introduction

The Hystrix Dashboard panel can be divided into upper and lower parts to query. The upper part is the information related to the circuit breaker method call, Circuit , and the lower part is the thread pool related information provided by Hystrix for the circuit breaker method, Thread Pools .

In the diagram, the circle in the upper left corner represents the flow and status of the method:

• The larger the circle, the greater the flow of the method
• The green circle represents the health of the circuit breaker, the yellow represents the sporadic failure of the circuit breaker, and the red represents the failure of the circuit breaker

Counter in upper right corner (three columns of numbers):
first column from top to bottom

• Green represents the current number of successful calls
• Blue represents the number of short circuit requests
• Turquoise represents the number of bad requests

Second column top to bottom

• Yellow represents the number of timed out requests
• Purple represents the number of thread pool rejections
• Red represents the number of failed requests

third column

• The percentage of bad requests in the past 10s

Thread Pools:
Hystrix will create a corresponding thread pool for a protected class. The purpose of this is that when the Hystrix command is called, it will not be affected by the method request thread (or the Hystrix worker thread and the caller thread. do not affect each other)

In the chart, the circle in the upper left corner represents the flow and status of the thread pool:

• The larger the circle, the more active the thread pool and the greater the flow
• The color of the circle represents the health of the thread pool

Bottom left from top to bottom:

• Active represents the number of active threads in the thread pool
• Queued represents the number of queued threads, this function is disabled by default, so it is always 0 by default
• Pool Size represents the number of threads in the thread pool

Bottom right from top to bottom:

• Max Active represents the maximum active thread, and the data displayed here is the maximum value of active threads in the current adoption cycle
• Execcutions represents the number of times threads in the thread pool are called to execute Hystrix commands
• Queue Size represents the size of the thread pool queue, disabled by default, meaningless