Microservice topic 10-Spring Cloud service circuit breaker

Preface

In the previous chapters, we talked about Spring Cloud load balancing and implemented client load balancing.

In this section, continue to share the content of the microservice topic, a total of 16 sections, namely:

• Microservice topic 01-Spring Application
• Microservice topic 02-Spring Web MVC view technology
• Microservice topic 03-REST
• Microservice topic 04-Spring WebFlux principle
• Microservice topic 05-Spring WebFlux application
• Microservices topic 06-Cloud Native Applications
• Microservice topic 07-Spring Cloud configuration management
• Microservice topic 08-Spring Cloud service discovery
• Microservice topic 09-Spring Cloud load balancing
• Microservice topic 10-Spring Cloud service circuit breaker
• Microservice topic 11-Spring Cloud service call
• Microservice topic 12-Spring Cloud Gateway
• Microservice topic 13-Spring Cloud Stream (on)
• Microservice topic 14-Spring Cloud Bus
• Microservice topic 15-Spring Cloud Stream implementation
• Microservice topic 16-Spring Cloud overall review

The key points of this section are:

• Spring Cloud Hystrix: As a short-circuit implementation of a server-side service, it introduces Spring Cloud Hystrix's common current limiting functions, and at the same time, explains the practical significance of health indicators and data indicators in a production environment
• Production preparation features: introduce the aggregate data indicators Turbine, Turbine Stream, and integrate Hystrix Dashboard

Fuse mechanism

Netflix Hystrix official wiki is
generally in a distributed system. When the client initiates a remote call to the service, if there are many clients, the concurrency of each server is limited. When it exceeds After the upper limit is accessed, the response of other clients may slow down or even fail. This is the so-called fusing, and there are usually two ways to achieve it.

• Control by timeout

Assuming that the number of thread pools provided by dubbo/http on the Service server is 200, the theoretically acceptable concurrency is 200. If the client's QPS exceeds 200, other clients will fall into a waiting state until there are idle threads in the thread pool .

The problem here is that the client's request time is unstable, some clients have a short request time, and some have a longer request time. It is obviously unreasonable for the server to wait for the slower client.

If the timeout period is set at this time, such as 200ms, regardless of whether the request is successful, abandon the timeout client and let the client in the queue enter the queue, which will greatly alleviate the "traffic congestion".

Q: What will the server usually return after fault tolerance through the timeout mechanism?
A: Regarding the problem of fault-tolerant return values: usually it can return null, or an empty object, or it can return an exception.

QPS: Queries Per Second means "query rate per second", which is the number of queries that a server can respond to per second, and is a measure of how much traffic a specific query server can handle within a specified time.
TPS: is the abbreviation of TransactionsPerSecond, which is the number of transactions per second. It is the unit of measurement for software test results. A transaction is a process in which a client sends a request to the server and the server responds. The client starts timing when sending a request, and ends timing after receiving a response from the server to calculate the time used and the number of completed transactions.

• Control by QPS counting.
  
  Usually in a distributed scenario, each server balances the number of QPS on the client by controlling the configuration of each server thread pool to achieve its goal.

Spring Cloud Hystrix Client

The content of the experiment is: if the set timeout time is greater than 100ms, the service will be blown.
We first quote Hystrix's configuration to implement service fusing.

Add annotations to the startup class:

@EnableHystrix               // 激活 Hystrix

Core realization:

 @HystrixCommand(
            fallbackMethod = "errorContent",
            commandProperties = {
    
    
                    @HystrixProperty(name = "execution.isolation.thread.timeoutInMilliseconds",
                            value = "100")
            }

    )
    @GetMapping("/say")
    public String say(@RequestParam String message) throws InterruptedException {
    
    
        // 如果随机时间 大于 100 ，那么触发容错
        int value = random.nextInt(200);

        System.out.println("say() costs " + value + " ms.");

        // > 100
        Thread.sleep(value);

        System.out.println("ServerController 接收到消息 - say : " + message);
        return "Hello, " + message;
    }

    public String errorContent(String message) {
    
    
        return "Fault";
    }

Implementation result test:
visit address: http://localhost:9090/say?message=test

At this time, the access display does not exceed 100ms, the processing procedure, and the printing information.

After multiple clicks, it is found that if the visit exceeds 100ms, it will be broken and the content after it will not be printed: The

following figure shows the data printed by the corresponding request:

How is the principle here realized? please watch the following part.

Fuse is not limited to Hystrix, can we implement such a method to achieve fuse?

Method signature

• Access qualifier

• Method return type

• Method name

• Method parameters

  • Number of methods
  • Method type + order
  • Method name (reserved when compiling, IDE, Debug)

Handwriting service circuit breaker (Future)

Low-level version (no fault tolerance implementation)

    private final ExecutorService executorService = Executors.newSingleThreadExecutor();

    /**
     * 简易版本
     *
     * @param message
     * @return
     * @throws InterruptedException
     */
    @GetMapping("/say2")
    public String say2(@RequestParam String message) throws Exception {
    
    
        Future<String> future = executorService.submit(() -> {
    
    
            return doSay2(message);
        });
        // 100 毫秒超时
        String returnValue = future.get(100, TimeUnit.MILLISECONDS);
        return returnValue;
    }

Experimental results test:

Visit address: http://localhost:9090/say2?message=test

The problem with this implementation is that there is no fault tolerance, that is, the program is still executed without any interception for more than 100ms.

Low-level version+ (with fault tolerance implementation)

    private final ExecutorService executorService = Executors.newSingleThreadExecutor();

    /**
     * 简易版本
     *
     * @param message
     * @return
     * @throws InterruptedException
     */
    @GetMapping("/say2")
    public String say2(@RequestParam String message) throws Exception {
    
    
        Future<String> future = executorService.submit(() -> {
    
    
            return doSay2(message);
        });
        // 100 毫秒超时
        String returnValue = null;
        try {
    
    
            returnValue = future.get(100, TimeUnit.MILLISECONDS);
        } catch (InterruptedException | ExecutionException | TimeoutException e) {
    
    
            // 超级容错 = 执行错误 或 超时
            returnValue = errorContent(message);
        }
        return returnValue;
    }

Experimental results test:

Visit address: http://localhost:9090/say2?message=test

Greater than 100ms:

Less than 100ms:

Intermediate version

Use the aop aspect to solve the problem:

1. First declare aop in the startup class:

@EnableAspectJAutoProxy(proxyTargetClass = true) // 激活 AOP

2. Web MVC configuration:

/**
 * Web MVC 配置
 */
@Configuration
public class WebMvcConfig implements WebMvcConfigurer {
    
    

    public void addInterceptors(InterceptorRegistry registry) {
    
    
        registry.addInterceptor(new CircuitBreakerHandlerInterceptor());
    }
}

3. Intermediate version

    /**
     * 中级版本
     *
     * @param message
     * @return
     * @throws InterruptedException
     */
    @GetMapping("/middle/say")
    public String middleSay(@RequestParam String message) throws Exception {
    
    
        Future<String> future = executorService.submit(() -> {
    
    
            return doSay2(message);
        });
        // 100 毫秒超时
        String returnValue = null;

        try {
    
    
            returnValue = future.get(100, TimeUnit.MILLISECONDS);
        } catch (TimeoutException e) {
    
    
            future.cancel(true); // 取消执行
            throw e;
        }
        return returnValue;
    }

4. Catch the timeout exception.

@RestControllerAdvice(assignableTypes = ServerController.class)
public class CircuitBreakerControllerAdvice {
    
    

    @ExceptionHandler
    public void onTimeoutException(TimeoutException timeoutException,
                                   Writer writer) throws IOException {
    
    
        writer.write(errorContent("")); // 网络 I/O 被容器
        writer.flush();
        writer.close();
    }

    public String errorContent(String message) {
    
    
        return "Fault";
    }

}

Experimental results test:

Visit address: http://localhost:9090/middle/say?message=test

Greater than 100ms:

Less than 100ms:

Advanced version (implementation without annotations)

1. Interface declaration

    /**
     * 高级版本
     *
     * @param message
     * @return
     * @throws InterruptedException
     */
    @GetMapping("/advanced/say")
    public String advancedSay(@RequestParam String message) throws Exception {
    
    
        return doSay2(message);
    }

2. ServerControllerAspect

@Aspect
@Component
public class ServerControllerAspect {
    
    

    private ExecutorService executorService = newFixedThreadPool(20);

    @Around("execution(* com.test.micro.services.spring.cloud." +
            "server.controller.ServerController.advancedSay(..)) && args(message) ")
    public Object advancedSayInTimeout(ProceedingJoinPoint point, String message) throws Throwable {
    
    
        Future<Object> future = executorService.submit(() -> {
    
    
            Object returnValue = null;
            try {
    
    
                returnValue = point.proceed(new Object[]{
    
    message});
            } catch (Throwable ex) {
    
    
            }
            return returnValue;
        });

        Object returnValue = null;
        try {
    
    
            returnValue = future.get(100, TimeUnit.MILLISECONDS);
        } catch (TimeoutException e) {
    
    
            future.cancel(true); // 取消执行
            returnValue = errorContent("");
        }
        return returnValue;
    }

    public String errorContent(String message) {
    
    
        return "Fault";
    }

    @PreDestroy
    public void destroy() {
    
    
        executorService.shutdown();
    }

}

Experimental results test:

http://localhost:9090/advanced/say?message=test

Greater than 100ms:

Less than 100ms:

Advanced version (implemented with annotations)

• TimeoutCircuitBreaker annotation

@Target(ElementType.METHOD) // 标注在方法
@Retention(RetentionPolicy.RUNTIME) // 运行时保存注解信息
@Documented
public @interface TimeoutCircuitBreaker {
    
    

    /**
     * 超时时间
     * @return 设置超时时间
     */
    long timeout();

}

• Aspect annotation implementation

    @Around("execution(* com.test.micro.services.spring.cloud." +
            "server.controller.ServerController.advancedSay2(..)) && args(message) && @annotation(circuitBreaker)")
    public Object advancedSay2InTimeout(ProceedingJoinPoint point,
                                        String message,
                                        CircuitBreaker circuitBreaker) throws Throwable {
    
    
        long timeout = circuitBreaker.timeout();
        return doInvoke(point, message, timeout);
    }

• Reflection API implementation

    @Around("execution(* com.test.micro.services.spring.cloud." +
            "server.controller.ServerController.advancedSay2(..)) && args(message) ")
    public Object advancedSay2InTimeout(ProceedingJoinPoint point,
                                        String message) throws Throwable {
    
    

        long timeout = -1;
        if (point instanceof MethodInvocationProceedingJoinPoint) {
    
    
            MethodInvocationProceedingJoinPoint methodPoint = (MethodInvocationProceedingJoinPoint) point;
            MethodSignature signature = (MethodSignature) methodPoint.getSignature();
            Method method = signature.getMethod();
            CircuitBreaker circuitBreaker = method.getAnnotation(CircuitBreaker.class);
            timeout = circuitBreaker.timeout();
        }
        return doInvoke(point, message, timeout);
    }

Experimental results test:

http://localhost:9090/advanced/say2?message=test

Greater than 100ms:

Less than 100ms:

Advanced version (semaphore realization = stand-alone current limiting solution)

@Target(ElementType.METHOD) // 标注在方法
@Retention(RetentionPolicy.RUNTIME) // 运行时保存注解信息
@Documented
public @interface SemaphoreCircuitBreaker {
    
    

    /**
     * 信号量
     *
     * @return 设置超时时间
     */
    int value();

}

    @Around("execution(* com.test.micro.services.spring.cloud." +
            "server.controller.ServerController.advancedSay3(..))" +
            " && args(message)" +
            " && @annotation(circuitBreaker) ")
    public Object advancedSay3InSemaphore(ProceedingJoinPoint point,
                                          String message,
                                          SemaphoreCircuitBreaker circuitBreaker) throws Throwable {
    
    
        int value = circuitBreaker.value();
        if (semaphore == null) {
    
    
            semaphore = new Semaphore(value);
        }
        Object returnValue = null;
        try {
    
    
            if (semaphore.tryAcquire()) {
    
    
                returnValue = point.proceed(new Object[]{
    
    message});
                Thread.sleep(1000);
            } else {
    
    
                returnValue = errorContent("");
            }
        } finally {
    
    
            semaphore.release();
        }

        return returnValue;

    }

Experimental results test:

http://localhost:9090/advanced/say3?message=test

Greater than 100ms:

By adding the semaphore, there will actually be a delay here, the so-called fusing, and then processing.

Less than 100ms:

postscript

Code address of this section: Hystrix

For more architectural knowledge, please pay attention to this series of articles on Java : The Road to Growth of Java Architects