Kubernetes Ingress Controller Introduction & Installation & Testing

Kubernetes Ingress Controller Introduction & Installation & Testing

basic introduction

For Kubernetes Service, both Cluster-Ip and NodePort are Layer 4 loads, how to achieve Layer 7 load balancing for services in the cluster. Ingress-nginx is a layer 7 load balancer, which is responsible for unified management of external requests for Services in the k8s cluster.

The first chapter will introduce the current mainstream Ingress Controller implementation. If you want to learn about Kubernetes Ingress Controller directly, please start from the second chapter.

1. Ingress Nginx selection

Refer to the blog for the content of this chapter.
At present, Ingress is the best way to expose services in the cluster. However, due to its important position, there are many Ingress Controllers in the world. The common ones are:

• Kubernetes Ingress
• Nginx Ingress
• Kong Ingress
• Traefik Ingress
• HAProxy Ingress
• Istio Ingress
• APISIX Ingress

In addition to those listed above, there are many Ingress Controllers. Faced with so many Ingress Controllers, how should we choose? What is the reference standard?

In general, it can be judged from the following dimensions:

• Supported protocols: whether to support protocols other than HTTP(S)
• Routing rules: what forwarding rules are there, and whether regularization is supported
• Deployment strategy: whether it supports ab deployment, canary deployment, blue-green deployment, etc.
• Upstream probe: By what mechanism is it used to determine whether the application is normal or not, whether there are active and passive checks, retries, fuses, custom health checks, and other solutions
• Load balancing algorithm: which load balancing algorithms are supported, Hash, session persistence, RR, WRR, etc.
• Authentication method: Which authorization schemes are supported? Basic, Digest, OAuth, External Authentication, etc.
• DDoS protection capability: whether it supports basic speed limit, white list, etc.
• Full link tracking: Can you access full link monitoring normally?
• Full link tracking: Can you access full link monitoring normally?
• JWT Validation: Is there a built-in JSON Web Token validation for authenticating and validating the user of the end application
• Graphical interface: Do you need a graphical interface
• Custom scalability: whether it is convenient to expand

The following is a brief introduction to the above-mentioned Ingress Controllers.

1.1 Kubernetes Ingress

The officially recommended Ingress controller of the official repository Kubernetes Ingress, which is based on the nginx web server and supplemented with a set of Lua plugins for additional functionality.

Due to the widespread use of Nginx, the Ingress controller is the easiest controller to use after migrating applications to K8S, and the learning cost is relatively low. If you do not require high controller capabilities, it is recommended to use it.

However, when there are too many configuration files, Reload is very slow, and although there are many plug-ins available, the plug-in expansion ability is very weak.

1.2 Nginx Ingress

The official repository
Nginx Ingress is the official version developed by NGINX, it is based on the commercial version of NGINX Plus, the NGINX controller has high stability, continuous backward compatibility, does not have any third-party modules, and is guaranteed due to the elimination of Lua code Higher speed (compared to official controller).

Compared with the official controller, it supports TCP/UDP traffic forwarding. The paid version has a wide range of additional functions. The main disadvantage is that it lacks authentication methods, traffic scheduling and other functions.

1.3 Kong Ingress

The official repository,
Kong Ingress, is built on top of NGINX with the addition of Lua modules that extend its functionality.

Kong used to focus on API gateways, but now it has become a mature Ingress controller. Compared with the official controller, it has improved routing matching rules, upstream probes, and authentication, and supports a large number of module plug-ins, and Convenience and configuration.

It provides some APIs and service definitions, which can be abstracted into Kubernetes CRDs, and the synchronization status can be completed to the Kong cluster through Kubernetes Ingress configuration.

1.4 Traefik Ingress

The official warehouse
traefik Ingress is a very comprehensive Ingress, officially called: Traefik is an Edge Router that makes publishing your services a fun and easy experience.

It has many useful features: continuous configuration update (without restart), support for multiple load balancing algorithms, web UI, metrics export, support for various protocols, REST API, Canary version, etc. Out-of-the-box "Let's Encrypt" support is another nice feature. Moreover, the version 2.0 already supports functions such as TCP/SSL, canary deployment, and traffic mirroring/shadowing, and the community is very active.

1.5 Istio Ingress

Istio , the official repository
, is a joint project of IBM, Google, and Lyft (the original author of Envoy), and it is a comprehensive service mesh solution. It not only manages all incoming external traffic (as an Ingress controller), but also controls all traffic inside the cluster. Behind the scenes, Istio uses Envoy as a secondary proxy for each service. Essentially, it's a big processor that can do pretty much anything. Its central idea is maximum control, scalability, security and transparency.

With Istio Ingress, you can fine-tune traffic routing, access authorization between services, balancing, monitoring, canary rollouts, and more.

However, the community now recommends using Ingress Gateways.

1.6 HAProxy Ingress

The official warehouse
HAProxy, as the ace load balancer, has the biggest advantage in load balancing among many controllers.

It provides "soft" configuration updates (no traffic loss), DNS-based service discovery, and dynamic configuration via API. HAProxy also supports fully custom configuration file templates (by replacing ConfigMap) and using Spring Boot functions in them.

1.7 APISIX Ingress

The official warehouse
ApiSix Ingress is an emerging Ingress Controller, which is mainly benchmarked against Kong Ingress.

It has very powerful routing capabilities, flexible plug-in expansion capabilities, and excellent performance. At the same time, its shortcomings are also very obvious. Although APISIX has a lot of functions after being open source, it lacks practical cases and relevant documents to guide you how to use these functions.

Attached is a comparison chart:

2. Kubernetes Ingress Controller

This section describes how to install the Kubernetes Ingress Controller. I will use the NodePort port exposure method to expose the ingress-nginx-controller service to the NodePort 80,443 port.

2.1 Kubernetes opens 1-65535 NodePort ports

If it is a cluster installed with kubeadm, the apiserver runs as a static pod, and the pod file is defined in /etc/kubernetes/manifests/kube-apiserver.yaml. The /etc/kubernetes/manifests directory is the definition of all static pod files. Kubelet will monitor the changes of the files in this directory. Whenever there is a change, the pod will be rebuilt and respond to the corresponding changes. So we modify the /etc/kubernetes/manifests/kube-apiserver.yaml file, and it will automatically take effect after adding the nodePort range parameter.

#编辑apiserver静态容器yaml文件
vi /etc/kubernetes/manifests/kube-apiserver.yaml

#在spec.containers.command下添加
- --service-node-port-range=1-65535

#完成后重启apiserver容器
kubectl delete pods -n kube-system -l component=kube-apiserver

2.2 Install Kubernetes Ingress Nginx

The official website guide
here adopts the installation method of the YAML manifest file. Before installation, pay attention to the version correspondence between the Kubernetes version and Kubernetes Ingress Nginx:

#切换到当前用户目录
cd ~

mkdir ingress-nginx

#下载安装用 manifest 文件
wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.7.1/deploy/static/provider/cloud/deploy.yaml

Edit deploy.yaml to modify the installation method to NodePort and specify a specific port

vi deploy.yaml 

#找到 ingress-nginx-controller 对应的service，可以通过在vi模式下搜索“LoadBalancer”关键字
修改为如下内容：
apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.7.1
  name: ingress-nginx-controller
  namespace: ingress-nginx
spec:
  externalTrafficPolicy: Local
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - appProtocol: http
    name: http
    nodePort: 80 #http开放80作为NodePort端口
    port: 80
    protocol: TCP
    targetPort: http
  - appProtocol: https
    name: https
    nodePort: 443 #https开放443作为NodePort端口
    port: 443
    protocol: TCP
    targetPort: https
  selector:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
  type: NodePort #服务方式设置为NodePort

Execute the installation with the modified deploy.yaml

kubectl apply -f deploy.yaml

#检查是否安装成功，看到一下状态则表示安装成功
[root@k8s-master ingress-controller]# kubectl get pods -n ingress-nginx
NAME                                        READY   STATUS      RESTARTS   AGE
ingress-nginx-admission-create-4cjmw        0/1     Completed   0          8h
ingress-nginx-admission-patch-5lqbc         0/1     Completed   1          8h
ingress-nginx-controller-6599b4f4c5-rkjlz   1/1     Running     0          7h51m

#查看相关的service
[root@k8s-master ingress-controller]# kubectl get svc -n ingress-nginx
NAME                                 TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                 AGE
ingress-nginx-controller             NodePort    172.16.80.22    <none>        80:80/TCP,443:443/TCP   8h
ingress-nginx-controller-admission   ClusterIP   172.16.85.159   <none>        443/TCP                 8h

3. Functional verification

Specifically divided into the following contents
1. Create a namespace named ingress-test for testing ingress
2. Create a deployment, run the nginx service, add the label: app=nginx
3. Create a service, port 80, and configure selector For app=nginx, used to point to nginx pod
4. Create an ingress, set service to name=nginx, port=80, used to point to nginx service, and configure the domain name test.nginx.com

#创建名为 test-nginx.yaml 的文件
vi test-nginx.yaml

#文件中添加如下内容
apiVersion: v1
kind: Namespace
metadata:
  name: ingress-test
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
  namespace: ingress-test
  labels:
    app: nginx
spec:
  selector:
    matchLabels:
      app: nginx
  replicas: 2
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: nginx
  namespace: ingress-test
spec:
  ports:
    - port: 80
      targetPort: 80
      name: nginx
  selector:
    app: nginx
  type: ClusterIP
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx
  namespace: ingress-test
  annotations: 
    kubernetes.io/ingress.class: "nginx"    # 指定 Ingress Controller 的类型
    nginx.ingress.kubernetes.io/use-regex: "true"    # 指定我们的 rules 的 path 可以使用正则表达式
spec:
  rules:
    - host: test.nginx.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: nginx
                port:
                  number: 80

After completion, deploy the above files and detect related content

kubectl apply -f test-nginx.yaml

#检查相关资源部署情况
[root@k8s-master ingress-controller]# kubectl get po,svc,ing -n ingress-test
NAME                         READY   STATUS    RESTARTS   AGE
pod/nginx-85996f8dbd-5t45h   1/1     Running   0          39s
pod/nginx-85996f8dbd-srs7b   1/1     Running   0          39s

NAME            TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE
service/nginx   ClusterIP   172.16.144.245   <none>        80/TCP    40s

NAME                                   CLASS    HOSTS            ADDRESS        PORTS   AGE
ingress.networking.k8s.io/nginx-test   <none>   test.nginx.com   172.16.80.22   80      39s

Access the deployed webpage through the local IP

#此时 curl —H 以及本机的ip后可以看到如下效果
[root@k8s-master ingress-controller]# curl -H "Host: test.nginx.com" http://<本机ip地址>
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
    
    
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

4. Set as the default Ingress Class (optional operation)

When creating an ingress, use spec.ingressClassName: to specify which IngressClass the ingress belongs to and which ingress controller it belongs to. Setting annotations ingressclass.kubernetes.io/is-default-class to true on the IngressClass resource will ensure that no specified ingressClassName New Ingress is assigned this default IngressClass.

#查看当前安装的 ingressclass，安装 Kubernetes Ingress Controller 默认的 ingressclass 名称是nginx
[root@k8s-cls-master ~]# kubectl get ingressclasses
NAME    CONTROLLER             PARAMETERS   AGE
nginx   k8s.io/ingress-nginx   <none>       2d5h

#编辑 ingressclasses 设置为默认，在metadata.annotations下添加：ingressclass.kubernetes.io/is-default-class: "true"
kubectl edit ingressclasses nginx
# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
  annotations:
    ingressclass.kubernetes.io/is-default-class: "true" #添加此行内容
    kubectl.kubernetes.io/last-applied-configuration: |
      {
    
    "apiVersion":"networking.k8s.io/v1","kind":"IngressClass","metadata":{
    
    "annotations":{
    
    },"labels":{
    
    "app.kubernetes.io/component":"controller","app.kubernetes.io/instance":"ingress-nginx","app.kubernetes.io/name":"ingress-nginx","app.kubernetes.io/part-of":"ingress-nginx","app.kubernetes.io/version":"1.7.1"},"name":"nginx"},"spec":{
    
    "controller":"k8s.io/ingress-nginx"}}
  creationTimestamp: "2023-05-27T04:08:12Z"
  generation: 1
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.7.1
  name: nginx
  resourceVersion: "3833131"
  uid: bcba9348-85bf-4b10-8492-60699a826c03
spec:
  controller: k8s.io/ingress-nginx