这个task描述弹性应用的熔断能力。熔断允许开发人员编写应用程序，以限制故障，延迟尖峰，网络特性造成的其他不良影响。这个task将展示如何为连接，请求，异常值检测配置熔断。
Before you begin
- 安装Istio
- 启动 httpbin 示例，并将它作为我们task的后端服务。
如果你启用了自动sidecar注入，执行
kubectl apply -f samples/httpbin/httpbin.yaml否则，你需要在启用httpbin 应用前，手动注入sidecar：
kubectl apply -f <(istioctl kube-inject -f samples/httpbin/httpbin.yaml)Circuit breaker
让我们建立一个方案来描述Istio的熔断能力。我们应该在之前部分运行过 httpbin 服务。
1.创建一个 destination rule 来指定我们在调用httpbin 服务时的熔断设置：
cat <<EOF | istioctl create -f -
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: httpbin
spec:
host: httpbin
trafficPolicy:
connectionPool:
tcp:
maxConnections: 1
http:
http1MaxPendingRequests: 1
maxRequestsPerConnection: 1
outlierDetection:
http:
consecutiveErrors: 1
interval: 1s
baseEjectionTime: 3m
maxEjectionPercent: 100
EOF2.验证我们刚刚创建的destination rule的正确性：
istioctl get destinationrule httpbin -o yamlapiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: httpbin
...
spec:
host: httpbin
trafficPolicy:
connectionPool:
http:
http1MaxPendingRequests: 1
maxRequestsPerConnection: 1
tcp:
maxConnections: 1
outlierDetection:
http:
baseEjectionTime: 180.000s
consecutiveErrors: 1
interval: 1.000s
maxEjectionPercent: 100Setting up our client
现在我们为调用 httpbin 服务已经创建了规则，现在让我们创建一个客户端来给我们的服务发送流量，并观察我们是否可以实现熔断。我们将使用一个荷载测试客户端 fortio 。用这个客户端，我们可以控制连接数，并发和外部HTTP调用的延迟。在这步，我们创建了一个注入了Istio sidecar proxy的客户端，这样我们的网络交互由Istio管理：
kubectl apply -f <(istioctl kube-inject -f samples/httpbin/sample-client/fortio-deploy.yaml)samples/httpbin/sample-client/fortio-deploy.yaml
现在我们应该能够进入到客户端pod中，使用示例fortio 工具去调用 httpbin. 我们会通过-curl来表明我们只想进行一次调用：
FORTIO_POD=$(kubectl get pod | grep fortio | awk '{ print $1 }')
kubectl exec -it $FORTIO_POD -c fortio /usr/local/bin/fortio -- load -curl http://httpbin:8000/getHTTP/1.1 200 OK
server: envoy
date: Tue, 16 Jan 2018 23:47:00 GMT
content-type: application/json
access-control-allow-origin: *
access-control-allow-credentials: true
content-length: 445
x-envoy-upstream-service-time: 36
{
"args": {},
"headers": {
"Content-Length": "0",
"Host": "httpbin:8000",
"User-Agent": "istio/fortio-0.6.2",
"X-B3-Sampled": "1",
"X-B3-Spanid": "824fbd828d809bf4",
"X-B3-Traceid": "824fbd828d809bf4",
"X-Ot-Span-Context": "824fbd828d809bf4;824fbd828d809bf4;0000000000000000",
"X-Request-Id": "1ad2de20-806e-9622-949a-bd1d9735a3f4"
},
"origin": "127.0.0.1",
"url": "http://httpbin:8000/get"
}你可以看到请求成功了！现在，让我们干点坏事~
Tripping the circuit breaker：
在熔断设置中，我们指定了 maxConnections: 1 和 httpMaxPendingRequests: 1. 这些意味着如果我们同时超过一个连接和请求，我们应该能看到istio-proxy为进一步请求/连接而开启电路。让我们尝试同时2个连接 (-c 2)并发送20个请求 (-n 20)
kubectl exec -it $FORTIO_POD -c fortio /usr/local/bin/fortio -- load -c 2 -qps 0 -n 20 -loglevel Warning http://httpbin:8000/getFortio 0.6.2 running at 0 queries per second, 2->2 procs, for 5s: http://httpbin:8000/get
Starting at max qps with 2 thread(s) [gomax 2] for exactly 20 calls (10 per thread + 0)
23:51:10 W http.go:617> Parsed non ok code 503 (HTTP/1.1 503)
Ended after 106.474079ms : 20 calls. qps=187.84
Aggregated Function Time : count 20 avg 0.010215375 +/- 0.003604 min 0.005172024 max 0.019434859 sum 0.204307492
# range, mid point, percentile, count
>= 0.00517202 <= 0.006 , 0.00558601 , 5.00, 1
> 0.006 <= 0.007 , 0.0065 , 20.00, 3
> 0.007 <= 0.008 , 0.0075 , 30.00, 2
> 0.008 <= 0.009 , 0.0085 , 40.00, 2
> 0.009 <= 0.01 , 0.0095 , 60.00, 4
> 0.01 <= 0.011 , 0.0105 , 70.00, 2
> 0.011 <= 0.012 , 0.0115 , 75.00, 1
> 0.012 <= 0.014 , 0.013 , 90.00, 3
> 0.016 <= 0.018 , 0.017 , 95.00, 1
> 0.018 <= 0.0194349 , 0.0187174 , 100.00, 1
# target 50% 0.0095
# target 75% 0.012
# target 99% 0.0191479
# target 99.9% 0.0194062
Code 200 : 19 (95.0 %)
Code 503 : 1 (5.0 %)
Response Header Sizes : count 20 avg 218.85 +/- 50.21 min 0 max 231 sum 4377
Response Body/Total Sizes : count 20 avg 652.45 +/- 99.9 min 217 max 676 sum 13049
All done 20 calls (plus 0 warmup) 10.215 ms avg, 187.8 qps我们看到几乎所有请求都通过了！
Code 200 : 19 (95.0 %)
Code 503 : 1 (5.0 %)
istio-proxy允许由一些偏差。让我们把同时连接数升到3：
kubectl exec -it $FORTIO_POD -c fortio /usr/local/bin/fortio -- load -c 3 -qps 0 -n 20 -loglevel Warning http://httpbin:8000/getFortio 0.6.2 running at 0 queries per second, 2->2 procs, for 5s: http://httpbin:8000/get
Starting at max qps with 3 thread(s) [gomax 2] for exactly 30 calls (10 per thread + 0)
23:51:51 W http.go:617> Parsed non ok code 503 (HTTP/1.1 503)
23:51:51 W http.go:617> Parsed non ok code 503 (HTTP/1.1 503)
23:51:51 W http.go:617> Parsed non ok code 503 (HTTP/1.1 503)
23:51:51 W http.go:617> Parsed non ok code 503 (HTTP/1.1 503)
23:51:51 W http.go:617> Parsed non ok code 503 (HTTP/1.1 503)
23:51:51 W http.go:617> Parsed non ok code 503 (HTTP/1.1 503)
23:51:51 W http.go:617> Parsed non ok code 503 (HTTP/1.1 503)
23:51:51 W http.go:617> Parsed non ok code 503 (HTTP/1.1 503)
23:51:51 W http.go:617> Parsed non ok code 503 (HTTP/1.1 503)
23:51:51 W http.go:617> Parsed non ok code 503 (HTTP/1.1 503)
23:51:51 W http.go:617> Parsed non ok code 503 (HTTP/1.1 503)
Ended after 71.05365ms : 30 calls. qps=422.22
Aggregated Function Time : count 30 avg 0.0053360199 +/- 0.004219 min 0.000487853 max 0.018906468 sum 0.160080597
# range, mid point, percentile, count
>= 0.000487853 <= 0.001 , 0.000743926 , 10.00, 3
> 0.001 <= 0.002 , 0.0015 , 30.00, 6
> 0.002 <= 0.003 , 0.0025 , 33.33, 1现在我们开始看到熔断按照我们的预期开始行动。
Code 200 : 19 (63.3 %)
Code 503 : 11 (36.7 %)
仅有 63.3% 的请求通过，剩下的被熔断捕获。我们可以通过查看istio-proxy的状态获取更多信息：
kubectl exec -it $FORTIO_POD -c istio-proxy -- sh -c 'curl localhost:15000/stats' | grep httpbin | grep pendingcluster.out.httpbin.springistio.svc.cluster.local|http|version=v1.upstream_rq_pending_active: 0
cluster.out.httpbin.springistio.svc.cluster.local|http|version=v1.upstream_rq_pending_failure_eject: 0
cluster.out.httpbin.springistio.svc.cluster.local|http|version=v1.upstream_rq_pending_overflow: 12
cluster.out.httpbin.springistio.svc.cluster.local|http|version=v1.upstream_rq_pending_total: 39我们看到 upstream_rq_pending_overflow 的值是12，这意味着目前为止熔断器被调用了12次。
Cleaning up
1.移除规则。
istioctl delete destinationrule httpbin2.关闭 httpbin 服务端和客户端。
kubectl delete deploy httpbin fortio-deploy
kubectl delete svc httpbin