kubernetes云原生纪元：领悟 Ingress Nginx(上)

文章目录

kubernetes云原生纪元：领悟 Ingress Nginx(上)

环境
DaemonSet部署ingress-nginx
tcp服务如何做一个服务发现

我们之前已经讲过 ingress nginx 基础知识，也成功部署到了kubernets中，但是真正在实际生产中应用还是会遇到很多问题。比对

用deployment 管理Ingress nginx-controller是否真的合适？
节点变化的控制是否非常方面？
我们的服务对外提供的tcp服务而不http服务怎么做服务发现呢？

服务超时？

服务对外提供https如何对外提供证书？

web服务有session保持怎么做？

小流量、AB测试怎么做？

这样的一些问题我们一一都去它解决掉。

环境

开始之前我们机器上要有ingress-nginx ,还有一个service ,两个deploy用于演示。

bluegreen-service.yaml

---
#service
apiVersion: v1
kind: Service
metadata:
  name: web-dev
  namespace: dev
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 8080
  selector:
    app: web-bluegreen
  type: ClusterIP

---
#ingress
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: web-bluegreen
  namespace: dev
spec:
  rules:
  - host: web-dev.mooc.com
    http:
      paths:
      - path: /
        backend:
          serviceName: web-dev
          servicePort: 80

We-bule.yaml

#deploy
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-blue
  namespace: dev
spec:
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  selector:
    matchLabels:
      app: web-bluegreen
  replicas: 2
  template:
    metadata:
      labels:
        app: web-bluegreen
        version: v1.0
    spec:
      containers:
      - name: web-bluegreen
        image: hub.mooc.com/kubernetes/web:v1
        ports:
        - containerPort: 8080

Web-green.yaml

#deploy
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-blue
  namespace: dev
spec:
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  selector:
    matchLabels:
      app: web-bluegreen
  replicas: 2
  template:
    metadata:
      labels:
        app: web-bluegreen
        version: v1.0
    spec:
      containers:
      - name: web-bluegreen
        image: hub.mooc.com/kubernetes/spring-boot-web:v1
        ports:
        - containerPort: 8080

解决第一个问题用deployment 管理Ingress nginx-controller是否真的合适？

DaemonSet部署ingress-nginx

我们之前使用deployment运行的，先查看下之前的

[root@master-001 ~]# kubectl get deploy -n ingress-nginx
NAME                       READY   UP-TO-DATE   AVAILABLE   AGE
nginx-ingress-controller   1/1     1            1           25d
[root@master-001 ~]# kubectl get deploy -n ingress-nginx nginx-ingress-controller -o yaml

我们可以考虑使用DaemonSet，好处这样不用考虑实例数了，我们需要在那个机器运行直接打标签，他就会自动跑起来，如果使用deployment 每次需要增加或者减少实例数的时候，需要调整 replicas。虽然也没什么问题，但是用起来没有那么顺畅。

把deployment 改成DaemonSet配置

首先把之前的配置保存成一个文件nginx-ingress-controller.yaml

[root@master-001 ~]# kubectl get deploy -n ingress-nginx nginx-ingress-controller -o yaml > nginx-ingress-controller.yaml

改成如下

[root@master-001 ~]# vi nginx-ingress-controller.yaml
apiVersion: apps/v1
kind: DaemonSet #
metadata:
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
  name: nginx-ingress-controller
  namespace: ingress-nginx
spec:
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/part-of: ingress-nginx
  updateStrategy:
    rollingUpdate:
      maxUnavailable: 1
    type: RollingUpdate
  template:
    metadata:
      annotations:
        prometheus.io/port: "10254"
        prometheus.io/scrape: "true"
      creationTimestamp: null
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
    spec:
      containers:
      - args:
        - /nginx-ingress-controller
        - --configmap=$(POD_NAMESPACE)/nginx-configuration
        - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
        - --udp-services-configmap=$(POD_NAMESPACE)/udp-services
        - --publish-service=$(POD_NAMESPACE)/ingress-nginx
        - --annotations-prefix=nginx.ingress.kubernetes.io
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.namespace
        image: registry.aliyuncs.com/google_containers/nginx-ingress-controller:0.26.1
        imagePullPolicy: Always
        lifecycle:
          preStop:
            exec:
              command:
              - /wait-shutdown
        livenessProbe:
          failureThreshold: 3
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
          initialDelaySeconds: 10
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 10
        name: nginx-ingress-controller
        ports:
        - containerPort: 80
          hostPort: 80
          name: http
          protocol: TCP
        - containerPort: 443
          hostPort: 443
          name: https
          protocol: TCP
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 10
        resources: {}
        securityContext:
          allowPrivilegeEscalation: true
          capabilities:
            add:
            - NET_BIND_SERVICE
            drop:
            - ALL
          runAsUser: 33
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      hostNetwork: true #使用host 网络
      nodeSelector:
        kubernetes.io/os: linux
        app: ingress #运行节点标签是app=ingress
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      serviceAccount: nginx-ingress-serviceaccount
      serviceAccountName: nginx-ingress-serviceaccount
      terminationGracePeriodSeconds: 300

删除之前的ingress-nginx

[root@master-001 ~]# kubectl delete deploy -n ingress-nginx nginx-ingress-controller
deployment.apps "nginx-ingress-controller" deleted

根据我们修改的配置文件创建新的ingress-nginx，可以看到正常运行

[root@master-001 ~]# kubectl apply -f nginx-ingress-controller.yaml
daemonset.apps/nginx-ingress-controller created
[root@master-001 ~]# kubectl get ds -n ingress-nginx
NAME                       DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
nginx-ingress-controller   1         1         1       1            1           kubernetes.io/os=linux   39s
[root@master-001 ~]# kubectl get pod -n ingress-nginx -o wide
NAME                             READY   STATUS    RESTARTS   AGE    IP               NODE       NOMINATED NODE   READINESS GATES
nginx-ingress-controller-s2nt5   1/1     Running   0          3m3s   172.16.126.139   node-001   <none>           <none>

访问下之前的应用，没问题

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-IE1PrPZm-1580869383636)(https://gitee.com/zhangchengji/kubeSource/raw/master/image/image-20200127214413627.png)]

如果我们想再其他节点运行一个ingress-nginx实例只需要在其他机器打一个标签app=ingress

[root@master-001 ~]# kubectl label node node-02 app=ingress

删除标签就自动停掉了

解决第二个问题我们的服务对外提供的tcp服务而不http服务怎么做服务发现呢？

tcp服务如何做一个服务发现

四层代理，

查看下ingress-nginx 的configmap

[root@master-001 ~]# kubectl get cm -n ingress-nginx
NAME                              DATA   AGE
ingress-controller-leader-nginx   0      25d
nginx-configuration               0      25d
tcp-services                      0      25d
udp-services                      0      25d

其中有一个 tcp-services,这个是在部署nginx 创建的

这里我们也自己做了一个tcp-services

这里我们就可以配置四层代理

30000 就是要暴露出的端口 dev/web-demo:80是对应具体的服务，也就是说web-demo:80暴露在ingress-nginx里面并且对外端口是3000

apiVersion: v1
kind: ConfigMap
metadata:
 name: tcp-services
 namespace: ingress-nginx
data: #30000 就是要暴露出的端口 dev/web-demo:80是对应具体的服务
 "30000": dev/web-demo:80 #dev命令空间下的service 名字是web-demo

创建一下

[root@master-001 ~]# kubectl apply -f tcp-config.yaml
configmap/tcp-services configured
[root@master-001 ~]# netstat -ntlp|grep 3000 #查看下监听端口

可以直接在浏览器访问IP:30000

当然web服务不需要开这种四层代理，主要用它来测试

这个web-demo 是指定service的名字

ZhangChengji/

发布了20 篇原创文章 · 获赞 3 · 访问量 587

私信关注