Kubernetes uses Ingress-nginx to implement the grayscale publishing function

Use Ingress to achieve grayscale publishing

Original link of this article: https://blog.csdn.net/xzk9381/article/details/109570832

1. Canary rule description

Ingress-Nginx is a K8S ingress tool that supports configuring Ingress Annotations to achieve grayscale release and testing in different scenarios (Ingress-Nginx is the Canary function introduced in version 0.21.0). Nginx Annotations supports the following 4 types of Canary rules:

nginx.ingress.kubernetes.io/canary-by-header: Traffic segmentation based on Request Header, suitable for grayscale release and A/B testing. When the Request Header is set always, the request will be sent up to the Canary version; when the Request Header is set never, the request will not be sent to the Canary inlet; for any other value Header, Header ignored, and the priority of the request by Other canary rules compare priorities.
nginx.ingress.kubernetes.io/canary-by-header-value: The value of the Request Header to be matched, used to notify the Ingress to route the request to the service specified in the Canary Ingress. When the Request Header is set to this value, it will be routed to the Canary entry. This rule allows users to customize the value of Request Header, which must be used together with the previous annotation (ie: canary-by-header).
nginx.ingress.kubernetes.io/canary-weight: Traffic segmentation based on service weight, suitable for blue-green deployment, with a weight range of 0-100 to route requests to the specified service in Canary Ingress in a percentage. A weight of 0 means that the canary rule will not send any requests to the service at the Canary portal. A weight of 100 means that all requests will be sent to the Canary portal.
nginx.ingress.kubernetes.io/canary-by-cookie: Cookie-based traffic segmentation, suitable for grayscale publishing and A/B testing. Cookie used to notify Ingress to route the request to the service specified in Canary Ingress. When the cookie value is set always, it will be routed to the inlet Canary; when the cookie value is set never, the request will not be sent to the Canary inlet; for any other value, and ignores the cookie request priority rules with other canaries Level comparison.

Note: The canary rules are sorted as follows in order of priority:

canary-by-header - > canary-by-cookie - > canary-weight

We can divide the above four annotation rules into the following two categories:

Canary rule based on weight:

Canary rules based on user request:

Two, deploy test cases

1. Deploy the official version service

First, create a deployment to represent the official version of the service, and write the yaml content as follows:

---
apiVersion: v1
kind: Namespace
metadata:
  name: ns-myapp
  labels:
    name: ns-myapp

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: production
  namespace: ns-myapp
spec:
  replicas: 1
  selector:
    matchLabels:
      app: production
  template:
    metadata:
      labels:
        app: production
    spec:
      containers:
      - name: production
        image: mirrorgooglecontainers/echoserver:1.10
        ports:
        - containerPort: 8080
        env:
          - name: NODE_NAME
            valueFrom:
              fieldRef:
                fieldPath: spec.nodeName
          - name: POD_NAME
            valueFrom:
              fieldRef:
                fieldPath: metadata.name
          - name: POD_NAMESPACE
            valueFrom:
              fieldRef:
                fieldPath: metadata.namespace
          - name: POD_IP
            valueFrom:
              fieldRef:
                fieldPath: status.podIP
---
apiVersion: v1
kind: Service
metadata:
  name: production
  namespace: ns-myapp
  labels:
    app: production
spec:
  ports:
  - port: 80
    targetPort: 8080
    protocol: TCP
    name: http
  selector:
    app: production

Create an Ingress routing rule for this service. The content of the yaml file is as follows:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: production
  namespace: ns-myapp
  annotations:
    kubernetes.io/ingress.class: nginx
spec:
  rules:
  - host: ingress.test.com
    http:
      paths:
      - backend:
          serviceName: production
          servicePort: 80

Apply the above yaml file and view the following information in k8s after creation:

[k8s-master ~]# kubectl get ingress -n ns-myapp
NAME         CLASS    HOSTS              ADDRESS        PORTS   AGE
production   <none>   ingress.test.com   10.16.13.201   80      4m25s

[k8s-master ~]# kubectl get pod -n ns-myapp
NAME                          READY   STATUS    RESTARTS   AGE
production-5698c4565c-jmjn5   1/1     Running   0          7m11s

At this point in the command line access ingress.test.comyou can see the following:

# curl ingress.test.com

Hostname: production-5698c4565c-jmjn5

Pod Information:
	node name:	dumlog013201
	pod name:	production-5698c4565c-jmjn5
	pod namespace:	ns-myapp
	pod IP:	10.42.0.74

Server values:
	server_version=nginx: 1.13.3 - lua: 10008

Request Information:
	client_address=10.16.13.201
	method=GET
	real path=/
	query=
	request_version=1.1
	request_scheme=http
	request_uri=http://ingress.test.com:8080/

Request Headers:
	accept=*/*
	host=ingress.test.com
	user-agent=curl/7.64.1
	x-forwarded-for=10.2.130.18
	x-forwarded-host=ingress.test.com
	x-forwarded-port=80
	x-forwarded-proto=http
	x-real-ip=10.2.130.18
	x-request-id=3019362be59228ee2284f5737fa39eb1
	x-scheme=http

Request Body:
	-no body in request-

Original link of this article: https://blog.csdn.net/xzk9381/article/details/109570832

2. Deploy Canary version service

Next, create a Canary version of the service to be used as a grayscale test.

Production versions of the above reference production.yamldocument, and then create a Canary version of the application, including a Canary version deploymentand service(for the convenience of a quick demonstration, just to production.yamlthe deployment and service keywords in the productiondirect replacement for the canaryactual scene may be involved in the business code change).

3. Canary rule test based on weight

The typical application scenario of weight-based traffic segmentation is 蓝绿部署that it can be implemented by setting the weight to 0 or 100. For example, you can set the Green version as the main part, and configure the entry of the Blue version as Canary. Initially, the weight is set to 0, so the traffic will not be proxied to the Blue version. Once the new version has been successfully tested and verified, the weight of the Blue version can be set to 100, that is, all traffic will be switched from the Green version to Blue.

The following canary.ingressare yaml file and then create an application-based routing Canary version of the weights (Ingress).

Note: To turn gray-release mechanism, you first need to set nginx.ingress.kubernetes.io/canary: "true"Canary, the Canary version of Ingress example enables the use of a weight-based traffic segmentation annotation rules will be allocated 30% of the traffic request to the Canary version.

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: canary
  namespace: ns-myapp
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "30"
spec:
  rules:
  - host: ingress.test.com
    http:
      paths:
      - backend:
          serviceName: canary
          servicePort: 80

Next, use the following command in the command line to access the domain name ingress.test.com 100 times, and calculate the proportion of each version's allocated traffic:

c=0;p=0;for i in $(seq 100); do result=$(curl -s ingress.test.com | grep  Hostname | awk -F: '{print $2}'); [[ ${result} =~ ^[[:space:]]canary ]] && let c++ || let p++; done;echo "production:${p}; canary:${c};"

The following results can be obtained:

production:73; canary:28;

4. Canary rule test based on user request

1. Based on Request Header

The typical application scenario of traffic segmentation based on Request Header is that 灰度发布或 A/B 测试场景.

Add an annotation to the Canary version of Ingress: nginx.ingress.kubernetes.io/canary-by-header: canary(the value of the annotation here can be any value), so that the current Ingress implementation is based on Request Header for traffic segmentation.

Modify the yaml file of the Canary version of Ingress to the following content:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: canary
  namespace: ns-myapp
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "30"
    nginx.ingress.kubernetes.io/canary-by-header: "canary"
spec:
  rules:
  - host: ingress.test.com
    http:
      paths:
      - backend:
          serviceName: canary
          servicePort: 80

Description: Canary rule to prioritize canary-by-header - > canary-by-cookie - > canary-weightthe following sort, so the above rules will be ignored ingress canary-weight of the original.

Due to the above rules ingress no canary-by-header: "canary"particular value is, i.e. nginx.ingress.kubernetes.io/canary-by-header-valuethe rules, so the time of the visit, only can be canaryassigned neveror always, when the header information canary:never, the request will not be sent to a canary channels; when the header information canary:always, the The request will always be sent to the canary version. Examples are as follows:

[k8s-master ~ ]# curl -s -H "canary:never" ingress.test.com | grep Hostname
Hostname: production-5698c4565c-jmjn5

[k8s-master ~ ]# curl -s -H "canary:always" ingress.test.com | grep Hostname
Hostname: canary-79c899d85-992nw

You can also add one on the basis of the previous annotation (canary-by-header) nginx.ingress.kubernetes.io/canary-by-header-value: user-value. Used to notify Ingress to route the matched request to the service specified in Canary Ingress.

Modify the yaml file of the Canary version of Ingress to the following content:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: canary
  namespace: ns-myapp
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "30"
    nginx.ingress.kubernetes.io/canary-by-header: "canary"
    nginx.ingress.kubernetes.io/canary-by-header-value: "true"
spec:
  rules:
  - host: ingress.test.com
    http:
      paths:
      - backend:
          serviceName: canary
          servicePort: 80

The above ingress rule sets the header information as canary:true, that is, the canary version is routed to only when the header value is satisfied. Examples are as follows:

[k8s-master ~ ]# curl -s ingress.test.com | grep Hostname
Hostname: production-5698c4565c-jmjn5

[k8s-master ~ ]# curl -s -H "canary:test" ingress.test.com | grep Hostname
Hostname: production-5698c4565c-jmjn5

[k8s-master ~ ]# curl -s -H "canary:true" ingress.test.com | grep Hostname
Hostname: canary-79c899d85-992nw

Five, Cookie-based Canary rule testing

Similar to the usage rules of annotation based on Request Header. For example A/B 测试场景, the need to allow local access Canary version Beijing users. Then when the cookie annotation is set to nginx.ingress.kubernetes.io/canary-by-cookie: "users_from_Beijing", the background can check the logged-in user request. If the user's access source is from Beijing, set the cookie users_from_Beijingvalue alwaysto ensure that users in Beijing only access the Canary version.

Modify the yaml file of the Canary version of Ingress to the following content:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: canary
  namespace: ns-myapp
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-by-cookie: "user_from_beijing"
spec:
  rules:
  - host: ingress.test.com
    http:
      paths:
      - backend:
          serviceName: canary
          servicePort: 80

An example of access is as follows:

[k8s-master ~ ]# curl -s -b "user_from_beijing=always" ingress.test.com | grep Hostname
Hostname: canary-79c899d85-992nw

[k8s-master ~ ]# curl -s -b "user_from_beijing=no" ingress.test.com | grep Hostname
Hostname: production-5698c4565c-jmjn5

Original link of this article: https://blog.csdn.net/xzk9381/article/details/109570832