本系列内容转载自git项目advancejava
Speaking in front of the request cache request buffer Hystrix, fallback graceful degradation, circuit breaker breaker fast-acting, this lecture, we explain in detail Hystrix thread pool limiting isolated interfaces.
Hystrix thread pool is full or by determining the amount of signal, exceeds the request capacity, go directly Reject degraded, so as to achieve a limiting role.
Current limit is to limit the amount of access to backend services, for example, you limit the resources MySQL, Redis, Zookeeper and various other back-end middleware access, in fact, it is to avoid too much traffic directly killed by the back-end services .
Design thread pool isolation technology
Hystrix using Bulkhead Partition bulkheads isolation technology to rely on external resource isolation, and thus avoid any malfunction of external dependencies cause the service to crash.
Isolation bulkhead, the interior space of the hull is to say the segment divided into several compartments, once a few compartments breakage water, the water will not flow therebetween each other, this way damaged when the ship still can have sufficient buoyancy and stability, and further reduce the risk of immediate wreck.
Hystrix dependence on external each with a separate thread pool, so, if reliance on the external call delay is very serious, is the most depleted that rely on their own thread pool only, does not affect other dependent calls.
Scene Hystrix application thread pool mechanism
• Each service will call dozens of back-end services depend on, rely on those back-end services are often composed of many different teams development.
• Each backend dependent services will provide its own client call the library, for example, with thrift, it would provide the corresponding thrift dependence.
• client call the library at any time to change.
• client 调用库随时可能会增加新的网络请求的逻辑。
• client 调用库可能会包含诸如自动重试、数据解析、内存中缓存等逻辑。
• client 调用库一般都对调用者来说是个黑盒，包括实现细节、网络访问、默认配置等等。
• 在真实的生产环境中，经常会出现调用者，突然间惊讶的发现，client 调用库发生了某些变化。
• 即使 client 调用库没有改变，依赖服务本身可能有会发生逻辑上的变化。
• 有些依赖的 client 调用库可能还会拉取其他的依赖库，而且可能那些依赖库配置的不正确。
• 大多数网络请求都是同步调用的。
• 调用失败和延迟，也有可能会发生在 client 调用库本身的代码中，不一定就是发生在网络请求中。
简单来说，就是你必须默认 client 调用库很不靠谱，而且随时可能发生各种变化，所以就要用强制隔离的方式来确保任何服务的故障不会影响当前服务。
线程池机制的优点
• 任何一个依赖服务都可以被隔离在自己的线程池内，即使自己的线程池资源填满了，也不会影响任何其他的服务调用。
• 服务可以随时引入一个新的依赖服务，因为即使这个新的依赖服务有问题，也不会影响其他任何服务的调用。
• 当一个故障的依赖服务重新变好的时候，可以通过清理掉线程池，瞬间恢复该服务的调用，而如果是 tomcat 线程池被占满，再恢复就很麻烦。
• 如果一个 client 调用库配置有问题，线程池的健康状况随时会报告，比如成功/失败/拒绝/超时的次数统计，然后可以近实时热修改依赖服务的调用配置，而不用停机。
• 基于线程池的异步本质，可以在同步的调用之上，构建一层异步调用层。
简单来说，最大的好处，就是资源隔离，确保说任何一个依赖服务故障，不会拖垮当前的这个服务。
线程池机制的缺点
• 线程池机制最大的缺点就是增加了 CPU 的开销。
除了 tomcat 本身的调用线程之外，还有 Hystrix 自己管理的线程池。
• 每个 command 的执行都依托一个独立的线程，会进行排队，调度，还有上下文切换。
• Hystrix 官方自己做了一个多线程异步带来的额外开销统计，通过对比多线程异步调用+同步调用得出，Netflix API 每天通过 Hystrix 执行 10 亿次调用，每个服务实例有 40 个以上的线程池，每个线程池有 10 个左右的线程。）最后发现说，用 Hystrix 的额外开销，就是给请求带来了 3ms 左右的延时，最多延时在 10ms 以内，相比于可用性和稳定性的提升，这是可以接受的。
我们可以用 Hystrix semaphore 技术来实现对某个依赖服务的并发访问量的限制，而不是通过线程池/队列的大小来限制流量。
semaphore 技术可以用来限流和削峰，但是不能用来对调研延迟的服务进行 timeout 和隔离。
execution.isolation.strategy 设置为 SEMAPHORE，那么 Hystrix 就会用 semaphore 机制来替代线程池机制，来对依赖服务的访问进行限流。如果通过 semaphore 调用的时候，底层的网络调用延迟很严重，那么是无法 timeout 的，只能一直 block 住。一旦请求数量超过了 semaphore 限定的数量之后，就会立即开启限流。
接口限流 Demo
Suppose a thread pool size is 8, the queue size of 10. When we set a long timeout longer, 20s.
In the internal command, dead code to write, to do a sleep, such as sleep 3s.
• withCoreSize: Set the thread pool size
• withMaxQueueSize: Set queue size
• withQueueSizeRejectionThreshold: and this withMaxQueueSizeconjunction, the size of the queue waiting to obtain is the smaller of these two parameters.
If only the size of the thread pool, two additional queue-related parameter is not set, the queue is in a closed state.
public class GetProductInfoCommand extends HystrixCommand<ProductInfo> {
private Long productId;
private static final HystrixCommandKey KEY = HystrixCommandKey.Factory.asKey("GetProductInfoCommand");
public GetProductInfoCommand(Long productId) {
super(Setter.withGroupKey(HystrixCommandGroupKey.Factory.asKey("ProductInfoService"))
.andCommandKey(KEY)
// 线程池相关配置信息
.andThreadPoolPropertiesDefaults(HystrixThreadPoolProperties.Setter()
// 设置线程池大小为8
.withCoreSize(8)
// 设置等待队列大小为10
.withMaxQueueSize(10)
.withQueueSizeRejectionThreshold(12))
.andCommandPropertiesDefaults(HystrixCommandProperties.Setter()
.withCircuitBreakerEnabled(true)
.withCircuitBreakerRequestVolumeThreshold(20)
.withCircuitBreakerErrorThresholdPercentage(40)
.withCircuitBreakerSleepWindowInMilliseconds(3000)
// 设置超时时间
.withExecutionTimeoutInMilliseconds(20000)
// 设置fallback最大请求并发数
.withFallbackIsolationSemaphoreMaxConcurrentRequests(30)));
this.productId = productId;
}
@Override
protected ProductInfo run() throws Exception {
System.out.println("调用接口查询商品数据，productId=" + productId);
if (productId == -1L) {
throw new Exception();
}
// 请求过来，会在这里hang住3秒钟
if (productId == -2L) {
TimeUtils.sleep(3);
}
String url = "http://localhost:8081/getProductInfo?productId=" + productId;
String response = HttpClientUtils.sendGetRequest(url);
System.out.println(response);
return JSONObject.parseObject(response, ProductInfo.class);
}
@Override
protected ProductInfo getFallback() {
ProductInfo productInfo = new ProductInfo();
productInfo.setName("降级商品");
return productInfo;
}
}
We simulated 25 requests. 8 before the request will be directly hang 3s when live call interface, then subsequent requests 10 will first wait for a request to enter the waiting queue in front of the execution is completed. The last seven requests over, will directly reject, call fallback downgrade logic.
@SpringBootTest
@RunWith(SpringRunner.class)
public class RejectTest {
@Test
public void testReject() {
for (int i = 0; i < 25; ++i) {
new Thread(() -> HttpClientUtils.sendGetRequest("http://localhost:8080/getProductInfo?productId=-2")).start();
}
// 防止主线程提前结束执行
TimeUtils.sleep(50);
}
}
From the results, we can clearly see that the print out of a total of 7 downgrade merchandise. This is the result of the request exceeds + number of queues and thread pools to directly reject.
ProductInfo(id=null, name=降级商品, price=null, pictureList=null, specification=null, service=null, color=null, size=null, shopId=null, modifiedTime=null, cityId=null, cityName=null, brandId=null, brandName=null)
ProductInfo(id=null, name=降级商品, price=null, pictureList=null, specification=null, service=null, color=null, size=null, shopId=null, modifiedTime=null, cityId=null, cityName=null, brandId=null, brandName=null)
ProductInfo(id=null, name=降级商品, price=null, pictureList=null, specification=null, service=null, color=null, size=null, shopId=null, modifiedTime=null, cityId=null, cityName=null, brandId=null, brandName=null)
ProductInfo(id=null, name=降级商品, price=null, pictureList=null, specification=null, service=null, color=null, size=null, shopId=null, modifiedTime=null, cityId=null, cityName=null, brandId=null, brandName=null)
ProductInfo(id=null, name=降级商品, price=null, pictureList=null, specification=null, service=null, color=null, size=null, shopId=null, modifiedTime=null, cityId=null, cityName=null, brandId=null, brandName=null)
ProductInfo(id=null, name=降级商品, price=null, pictureList=null, specification=null, service=null, color=null, size=null, shopId=null, modifiedTime=null, cityId=null, cityName=null, brandId=null, brandName=null)
调用接口查询商品数据，productId=-2
调用接口查询商品数据，productId=-2
调用接口查询商品数据，productId=-2
调用接口查询商品数据，productId=-2
调用接口查询商品数据，productId=-2
调用接口查询商品数据，productId=-2
调用接口查询商品数据，productId=-2
调用接口查询商品数据，productId=-2
ProductInfo(id=null, name=降级商品, price=null, pictureList=null, specification=null, service=null, color=null, size=null, shopId=null, modifiedTime=null, cityId=null, cityName=null, brandId=null, brandName=null)
{"id": -2, "name": "iphone7手机", "price": 5599, "pictureList":"a.jpg,b.jpg", "specification": "iphone7的规格", "service": "iphone7的售后服务", "color": "红色,白色,黑色", "size": "5.5", "shopId": 1, "modifiedTime": "2017-01-01 12:00:00", "cityId": 1, "brandId": 1}
// 后面都是一些正常的商品信息，就不贴出来了