1. Problems faced by distributed systems
Service Avalanche
When calling between multiple services, it is assumed that microservice A calls microservice B and microservice C, and microservice B and microservice C call other microservices, which is the so-called "fan-out". If the response time of a microservice call on the fan-out link is too long or unavailable, the call to microservice A will occupy more and more system resources, causing the system to crash, which is the so-called "avalanche effect" "
For high-traffic applications, a single backend dependency can cause all resources on all servers to saturate within seconds. Worse than failure, these applications can also cause increased latency between services, strain backup queues, threads, and other system resources, leading to more cascading failures across the system. These all represent the need to isolate and manage failures and latencies so that the failure of a single dependency cannot take down the entire application or system
2. Overview
Hystrix is an open source library for dealing with the delay and fault tolerance of distributed systems. In distributed systems, many dependencies will inevitably fail to call, such as timeouts and exceptions. Hystrix can ensure that in the case of a dependency problem, no It will lead to the failure of the overall service, avoid cascading failures, and improve the resilience of the distributed system.
"Short-circuit by" itself is a switching device. When a service unit fails, it returns an expected and processable alternative response to the caller through fault monitoring of the disconnected route (similar to blowing a fuse) . Instead of waiting for a long time or throwing exceptions, this ensures that the thread of the service caller will not be occupied for a long time and unnecessarily, thus avoiding the spread of faults in the distributed system, and even avalanches.
The circuit breaker mechanism is a protection mechanism for microservice links to deal with the avalanche effect.
When the response time of a microservice call on the "fan-out" link is too long, the service will be downgraded, and then the call of the node's microservice will be blown, and the "error" response information will be quickly returned. When it is detected that the node microservice invocation responds normally and restores the invocation link. In the Spring Cloud framework, the circuit breaker mechanism is implemented through Hystrix. Hystrix will monitor the call status between microservices. When the failed call reaches a certain threshold, the default value is that if 20 calls fail within 5s, the circuit breaker mechanism will be activated. The annotation for the circuit breaker is @HystrixCommand.
3. Service fuse
provider code
@RestController public class DeptController { @Autowired private DeptService service = null; @RequestMapping(value = "/dept/get/{id}", method = RequestMethod.GET) // Once the service method fails and an error message is thrown, it will automatically call the fallbackMethod call class marked by @HystrixCommand Specified method @HystrixCommand(fallbackMethod = "processHystrix_Get" ) public Dept get(@PathVariable("id" ) Long id) { Dept dept = this.service.get(id); if ( null == dept) { throw new RuntimeException("The ID:" + id + "There is no corresponding information"); // When an exception is thrown in this place, it will trigger @HystrixCommand } return dept; } public Dept processHystrix_Get(@PathVariable("id") Long id) { return new Dept().setDeptno(id).setDname("The ID: " + id + "There is no corresponding information, null--@HystrixCommand" ) .setDb_source("no this database in MySQL"); } }
4. Service downgrade
The overall resources are almost not enough, so I reluctantly shut down some services first, and then reopened after the difficulties were overcome.
Service downgrade processing is implemented on the client side and has nothing to do with the server side
V. Summary
Service fuse:
Usually caused by a service failure or abnormality, similar to the "fuse" in reality, when an abnormal condition is triggered, the entire service is directly blown, instead of waiting until the service times out
Service downgrade
The so-called downgrade is generally considered from the overall load. That is, when a service is broken, the server will no longer be called. At this time, the client can prepare a local fallback callback and return a default value. In this way, although the service level is reduced, it can be used anyway, which is better than directly hanging up.