在kubernetes中使用nginx ingress实现灰度部署

概述

1）kuberntes集群环境：v1.14.2。
2）nginx ingress版本：nginx-ingress-controller:0.25.0。
3）nginx ingress部署方式：daemonset，并直接使用宿主网络（即host模式），80端口。
4）官方文档地址：https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#canary

1）基于权重的灰度发布（30%请求流入canary服务）

1）步骤1：部署production和canary服务

[root@master canary]# cat production.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: production
  labels:
    app: production
spec:
  replicas: 1
  selector:
    matchLabels:
      app: production
  template:
    metadata:
      labels:
        app: production
    spec:
      containers:
      - name: production
        image: registry.aliyuncs.com/google_containers/echoserver:1.10
        ports:
        - containerPort: 8080
        env:
          - name: NODE_NAME
            valueFrom:
              fieldRef:
                fieldPath: spec.nodeName
          - name: POD_NAME
            valueFrom:
              fieldRef:
                fieldPath: metadata.name
          - name: POD_NAMESPACE
            valueFrom:
              fieldRef:
                fieldPath: metadata.namespace
          - name: POD_IP
            valueFrom:
              fieldRef:
                fieldPath: status.podIP
---
apiVersion: v1
kind: Service
metadata:
  name: production
  labels:
    app: production
spec:
  ports:
  - port: 80
    targetPort: 8080
    protocol: TCP
    name: http
  selector:
    app: production


[root@master canary]# cat canary.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: canary
  labels:
    app: canary
spec:
  replicas: 1
  selector:
    matchLabels:
      app: canary
  template:
    metadata:
      labels:
        app: canary
    spec:
      containers:
      - name: canary
        image: registry.aliyuncs.com/google_containers/echoserver:1.10
        ports:
        - containerPort: 8080
        env:
          - name: NODE_NAME
            valueFrom:
              fieldRef:
                fieldPath: spec.nodeName
          - name: POD_NAME
            valueFrom:
              fieldRef:
                fieldPath: metadata.name
          - name: POD_NAMESPACE
            valueFrom:
              fieldRef:
                fieldPath: metadata.namespace
          - name: POD_IP
            valueFrom:
              fieldRef:
                fieldPath: status.podIP
---
apiVersion: v1
kind: Service
metadata:
  name: canary
  labels:
    app: canary
spec:
  ports:
  - port: 80
    targetPort: 8080
    protocol: TCP
    name: http
  selector:
    app: canary

2）访问product和canary服务以检查服务是否正常运行

3）部署production和canary服务对应的ingress

[root@master canary]# cat production-ingress.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: production
  annotations:
    kubernetes.io/ingress.class: nginx
spec:
  rules:
  - host: master.example.com
    http:
      paths:
      - backend:
          serviceName: production
          servicePort: 80
        path: /apis/echo

[root@master canary]# cat canary-weight-ingress.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: canary
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "30"
spec:
  rules:
  - host: master.example.com
    http:
      paths:
      - backend:
          serviceName: canary
          servicePort: 80
        path: /apis/echo

4）访问nginx查看灰度部署效果

2）基于http header的灰度发布

#在已经成功部署production服务和canary服务的基础上，创建以下两个ingress
[root@master canary]# cat production-ingress.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: production
  annotations:
    kubernetes.io/ingress.class: nginx
spec:
  rules:
  - host: master.ly-sky.com
    http:
      paths:
      - backend:
          serviceName: production
          servicePort: 80
        path: /apis/echo


[root@master canary]# cat canary-header-ingress.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: canary
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-by-header: "yourcanaryheader"
    nginx.ingress.kubernetes.io/canary-by-header-value: "boy"
spec:
  rules:
  - host: master.ly-sky.com
    http:
      paths:
      - backend:
          serviceName: canary
          servicePort: 80
        path: /apis/echo

4）访问nginx查看灰度部署效果

根据ingress中的设置，请求带http header "yourcanaryheader: boy"的请求，会流入canary服务。

当yourcanaryheader这个http header的值不是"boy"时，请求是流入product服务。