Use traefik reverse proxy k8s dashboard

Production environment, k8s cluster of external exposure and Ingress LoadBalancer services are mainly in two ways:

LoadBalancer: Cloud vendors need to support, use k8s service load balancing capabilities, that is, relying on iptables / ipvs ability, it can be used for a variety of protocols
Ingress: relatively more flexible, load balancing through the reverse proxy server, used only for http / https protocol, the need for additional services as well as reverse proxy ingress controller In this scenario, nginx reverse proxy is well-known, in the era of k8s , there has been nginx-ingress , is nginx + ingress combination of controller, ingress controller is responsible for generating nginx configuration of the ingress resource, when the configuration changes is to restart nginx. There has also been a cloud native reverse proxy traefik, which corresponds to the ingress controller which comprises as one, and can dynamically change aware routing rules, need to restart.

traefik is a relatively new reverse proxy, Internet-related information is not particularly rich, study for several days before a successful visit to k8s dashboard, will be one of the key points in this record.

Installation traefik

Use helm installation, traefik 1.7.19 latest chart used:

helm install stable/traefik -f traefik-values.yaml

traefik-values.yaml:

rbac:
  enabled: true
dashboard:
  enabled: true # 启用traefik dashboard
  ingress:
    annotations:
      traefik.ingress.kubernetes.io/rule-type: PathPrefixStrip
deployment:
  hostPort:
    httpEnabled: true # traefik pod所在node上开启80端口
    httpsEnabled: true # traefik pod所在node上开启443端口
    dashboardEnabled: true # traefik pod所在node上开启8080端口，共traefik dashboard使用
ssl:
  insecureSkipVerify: true # frontend不验证https的benkend
  enabled: true # 启用https入口
extraVolumes:
  - name: traefik-ssl
    hostPath:
      path: /share/k8s/traefik/ssl # 其中存放https入口的证书和key，名字必须为tls.crt,tls.key
      type: DirectoryOrCreate
extraVolumeMounts:
  - name: traefik-ssl
    mountPath: /ssl   # traefik pod从/ssl目录读取上述tls.crt,tls.key

Detailed configuration methods, see the official documents , the above-mentioned key points are as follows:

Https inlet opening provided ssl.enabled = true, then a certificate and key, the above-mentioned node by node from the local directory mount the pod to the embodiment, each node first node and put certificate key, better way by k8s secret, then mount the pod to create a secret
How to access to the entrance, I was through the open port on the node, then you can access to the entrance through the node pod where, through http: // nodeip or https: // nodeip; you can also use NodePort type of service, so at http: // any-nodeip: http-nodeport or https: // any-nodeip: https-nodeport access, value set serviceType: nodePort
PathPrefixStrip routes matching rules I use the default name is the host match

Because traefik dashboard is enabled, the installation will automatically create a dashboard traefik of ingress:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    traefik.ingress.kubernetes.io/rule-type: PathPrefixStrip
  labels:
    app: traefik
    chart: traefik-1.82.1
    heritage: Tiller
    release: traefik
  name: traefik-dashboard
  namespace: default
spec:
  rules:
  - host: traefik.example.com
    http:
      paths:
      - backend:
          serviceName: traefik-dashboard
          servicePort: dashboard-http

traefik by label app: Select the required perception of ingress traefik. You added ingress note contains the tag. Above annotations and host are derived from the value. Because I do not want with host, so use PathPrefixStrip routing rules, I modified the above ingress as follows:

spec:
  rules:
  - http:
      paths:
      - backend:
          serviceName: traefik-dashboard
          servicePort: dashboard-http
        path: /traefik

So that when using http: // nodeip / traefik can access the dashboard, because the node also opens the dashboard port, or through http: // nodeip: 8080 visit.

Acting k8s dashboard

The latest k8s dashboard (v2.0.0-beta6) mounted kubernetes-dashboard namespace:

kubectl get svc -n kubernetes-dashboard
NAME                        TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)         AGE
dashboard-metrics-scraper   ClusterIP   10.254.238.13    <none>        8000/TCP        21d
kubernetes-dashboard        LoadBalancer   10.254.253.226   <pending>     443:30223/TCP   21d

Increased ingress:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    traefik.ingress.kubernetes.io/rule-type: PathPrefixStrip
  labels:
    app: traefik
  name: kubernetes-dashboard
  namespace: default
spec:
  rules:
  - http:
      paths:
      - backend:
          serviceName: kubernetes-dashboard
          servicePort: 443
        path: /k8s

Https proxy backend

k8s dashboard only supports https access, the first question is how stuck https proxy service, frontend to backend routing will appear the following situations:

http->http
http->https
https->http
https->https

When the backend is https, whether frontend http or https, that is, 2 and 4, will be reported 500 errors, can not be verified because the frontend backend, this time Solution:

Either set insecureSkipVerify, this is relatively simple, if this is always the best way frontend using https, which is set redirect
Either set the ingress tls, configure host of tls certificate information

Set insecureSkipVerify method I used. Generally the best way is to use the entrance always use https, then end tls, whether the backend https unimportant.

Acting different namespace service

After solving the above problems, the next encounter k8s dashboard service can not access issues, in traefik dashboard displays in red, because the helm traefik installed by default in the default namespace rather k8s dashboard mounted kubernetes-dashboard namespace, you can not access across namespace to service, the solution:

Either the traefik installed in the same space and k8s dashboard

Either the dashboard service into the default namespace by ExternalName

apiVersion: v1
kind: Service
metadata:
 name: kubernetes-dashboard
 namespace: default
spec:
 ports:
 - name: https
   port: 443
   protocol: TCP
   targetPort: 443
 sessionAffinity: None
 type: ExternalName
 externalName: kubernetes-dashboard.kubernetes-dashboard.svc.cluster.local

ExternalName method I used. The full domain name service is servicename.namespace.svc.cluster.local, cluster.local kubelet is configured.

Based on path routing

Services can be visited, but appeared MIME type is not a supported stylesheet MIME type errors.

Initially I thought it was traefik joined the X-Content-Type-Options in reponse header in: nosniff, but found traefik default is not added.

Finally found url path, I use only the ingress path routing, do not use host.

When using https: when // nodeip / k8s access k8s dashboard, because the routing rules are PathPrefixStrip, to the back-end request is https: // nodeip, then get home, the file name is k8s, the main page k8s in css, js, etc. file path is relative to the current document path, so the request url is https: //nodeip/xxx.css, then you do not match the rules of the road, the above error.

If you use https: // nodeip / k8s / access dashboard, on all normal.

So there is a certain risk when using path matching the route, and resource path in the definition of home:

Home page css, js and other resource path defined way	Explanation
Not defined base, not the path to resources ./ or ../ and / or at the beginning or `<base href="./">`resource path beginning with ./	1. Match / path, only through https: // xxxx / path / access 2. Match / path, and redirected to a rear end sub /, this time through https: // xxxx / path / or https: // xxxx / path can be accessed
`<base href="/">`, Resource path not to ./ or ../ or / beginning	Only match / matches other paths are not working really encountered such a situation, such as monocular

So the best way to match or host route.

dashboard certification

After understanding the above-mentioned problems, and finally into the dashboard login screen:

At first I was using http entrance, using Token landing approach, there is no response, through the development of tools to view, find the problem in the use http entrance, header does not carry jweToken, authentication fails, you must use https entrance.

Recall before the adoption of kubectl proxy, that is http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/also not landing, in fact, is the same problem.

Use http entrance landing failed:

When using the https entry, jweToken is carrying, landing success:

)

So decisive set frontend always https, values increase traefik.ingress.kubernetes.io/redirect-entry-point: https, then helm upgrade, increase the ingress own needs its own modification:

dashboard:
  enabled: true
  ingress:
    annotations:
      traefik.ingress.kubernetes.io/rule-type: PathPrefixStrip
      traefik.ingress.kubernetes.io/redirect-entry-point: https
      # 不要使用ingress.kubernetes.io/ssl-redirect: "true"，因为会丢掉path

Whether such use http: // nodeip / k8s / or https: // nodeip / k8s / can be successfully landed.

Welcome to Zhong Pan's blog