k8s上部署traefik

traefik部署

this section shows how to deploy traefik on kubernetes.

官方主要定义了两种部署模式，分别是Daemonset和Deployment，它们之间的区别主要是：

• 相比一个节点只部署一个daemonset的traefik，采用deployment会更易于伸缩和扩展；
• Daemonset可以利用taints和tolerations字段在自定义的节点上部署traefik服务；也可以给节点打标签，使用nodeSelector。
• 采用Daemonset方式，可以在任何节点上访问80和443端口(hostwork模式下才可以)，而使用deployment必须依赖service里面定义的对象去访问。

rbac 授权

Kubernetes在1.6+中引入了基于角色的访问控制（RBAC），以允许对Kubernetes资源和API进行细粒度控制。

如果您的集群配置了RBAC，则需要授权Traefik使用Kubernetes API。 有两种方法可以设置适当的权限：通过特定于ns的RoleBindings或单个全局ClusterRoleBinding.

每个ns的RoleBinding可以限制授予权限，只有Traefik正在监视的ns才能使用，从而遵循最小权限原则。 如果Traefik不应该监视所有ns，并且ns不会动态更改，那么使用RoleBinding。 否则，必须使用单个ClusterRoleBinding

---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
rules:
  - apiGroups:
      - ""
    resources:
      - services
      - endpoints
      - secrets
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
    - extensions
    resources:
    - ingresses/status
    verbs:
    - update
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: traefik-ingress-controller
subjects:
- kind: ServiceAccount
  name: traefik-ingress-controller
  namespace: kube-system

http方式的ds部署

使用ds的方式部署traefik，ds可以使用 NET_BIND_SERVICE 功能运行，这将允许它绑定到每个主机上的端口80/443 / etc。 这将允许绕过kube-proxy，并减少流量跳跃。

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: traefik-ingress-controller
  namespace: kube-system
---
kind: DaemonSet
apiVersion: extensions/v1beta1
metadata:
  name: traefik-ingress-controller
  namespace: kube-system
  labels:
    k8s-app: traefik-ingress-lb
spec:
  template:
    metadata:
      labels:
        k8s-app: traefik-ingress-lb
        name: traefik-ingress-lb
    spec:
      serviceAccountName: traefik-ingress-controller
      terminationGracePeriodSeconds: 60
      containers:
      - image: traefik
        name: traefik-ingress-lb
        ports:
        - name: http
          containerPort: 80
          hostPort: 80
        - name: admin
          containerPort: 8080
        securityContext:
          capabilities:
            drop:
            - ALL
            add:
            - NET_BIND_SERVICE
        args:
        - --api
        - --kubernetes
        - --logLevel=INFO
---
kind: Service
apiVersion: v1
metadata:
  name: traefik-ingress-service
  namespace: kube-system
spec:
  selector:
    k8s-app: traefik-ingress-lb
  ports:
    - protocol: TCP
      port: 80
      name: web
    - protocol: TCP
      port: 8080
      name: admin

此配置我的环境需要配置，你们可略过，
同样traefik部署和nginx controller一样，也需要配置如下env:

        env:
            - name: KUBERNETES_SERVICE_HOST
              value: "172.18.12.241"
            - name: KUBERNETES_SERVICE_PORT
              value: "443"

测试traefik ui

使用 curl nodeip 访问发现404，这是因为traefik默认是给ingress提供的80端口，而自己的ui是8080端口。

//部署svc和ingress
通过 ui.tr  或 nodeip +8080即可访问
apiVersion: v1
kind: Service
metadata:
  name: traefik-web-ui
  namespace: kube-system
spec:
  selector:
    k8s-app: traefik-ingress-lb
  ports:
  - name: web
    port: 80
    targetPort: 8080
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: traefik-web-ui
  namespace: kube-system
spec:
  rules:
  - host: ui.tr
    http:
      paths:
      - path: /
        backend:
          serviceName: traefik-web-ui
          servicePort: web

测试ingress

创建ingress, 并创建 label-service 的 svc

//cat  my-ingress.test.yml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: my-ingress
spec:
  rules:
  - host: my.example.com
    http:
      paths:
      - backend:
          serviceName: label-service
          servicePort: 80

在 /etc/hosts中配置：traefikip my.example.com

https方式部署traefik(向入口添加证书)

1）提供证书：

//使用openssl命令生成ca证书:
openssl req -newkey rsa:2048 -nodes -keyout tls.key -x509 -days 36500 -out tls.crt

2）创建secret保存证书和私钥

kubectl create secret generic traefik-cert --from-file=tls.crt --from-file=tls.key -n kube-system

3）创建configmap：

cat traefik.toml
traefik 配置文件（从http访问跳转到https,证书及私钥放在/ssl/目录下，需要secret挂载到该目录供traefik读取）

defaultEntryPoints = ["http", "https"]
insecureSkipVerify = true
[entryPoints]
  [entryPoints.http]
  address = ":80"
    [entryPoints.http.redirect]
      entryPoint = "https"
  [entryPoints.https]
  address = ":443"
    [entryPoints.https.tls]
      [[entryPoints.https.tls.certificates]]
      CertFile = "/ssl/tls.crt"
      KeyFile = "/ssl/tls.key"


 kubectl create configmap traefik-conf --from-file=traefik.toml -n kube-system

4）重新创建traefik

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: traefik-ingress-controller
  namespace: kube-system
---
kind: DaemonSet
apiVersion: extensions/v1beta1
metadata:
  name: traefik-ingress-controller
  namespace: kube-system
  labels:
    k8s-app: traefik-ingress-lb
spec:
  template:
    metadata:
      labels:
        k8s-app: traefik-ingress-lb
        name: traefik-ingress-lb
    spec:
      nodeSelector:
        traefik: "svc"
      serviceAccountName: traefik-ingress-controller
      terminationGracePeriodSeconds: 60
      volumes:
      - name: ssl
        secret:
          secretName: traefik-cert
      - name: config
        configMap:
          name: traefik-conf
      containers:
      - image: traefik
        name: traefik-ingress-lb
        volumeMounts:
        - mountPath: "/ssl"
          name: ssl
        - mountPath: "/config"
          name: config
        env:
          - name: KUBERNETES_SERVICE_HOST
            value: "172.18.12.241"
          - name: KUBERNETES_SERVICE_PORT
            value: "443"
        ports:
        - name: http
          containerPort: 80
          hostPort: 80
        - name: https
          containerPort: 443
          hostPort: 443
        - name: admin
          containerPort: 8080
          hostPort: 8080
        securityContext:
          capabilities:
            drop:
            - ALL
            add:
            - NET_BIND_SERVICE
        args:
        - --configfile=/config/traefik.toml
        - --api
        - --kubernetes
        - --logLevel=DEBUG

5)更新Traefik Service，开放443端口：

kind: Service
apiVersion: v1
metadata:
  name: traefik-ingress-service
  namespace: kube-system
spec:
  selector:
    k8s-app: traefik-ingress-lb
  ports:
    - protocol: TCP
      port: 80
      name: web
    - protocol: TCP
      port: 8080
      name: admin
    - protocol: TCP
      port: 443
      name: tls

6. 创建ingress

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
 name: traefik-web-ui
 namespace: kube-system
 annotations:
   kubernetes.io/ingress.class: traefik
spec:
 rules:
 - host: ui.domain.com
   http:
     paths:
     - path: /
       backend:
         serviceName: traefik-ingress-service
         servicePort: admin
 tls:
 - hosts:
   - ui.domain.com
   secretName: traefik-cert 

或者：tls的hosts将会被忽略。

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
 name: traefik-web-ui
 namespace: kube-system
 annotations:
   kubernetes.io/ingress.class: traefik
spec:
 rules:
 - host: ui.domain.com
   http:
     paths:
     - path: /
       backend:
         serviceName: traefik-ingress-service
         servicePort: admin
 tls:
 - secretName: traefik-cert

7. 测试访问：
   都可通过http和https访问

可以通过  https://ui.domain.com  和 http://ui.domain.com 访问traefik ui

其他tls 问题

不同的 ingress 对象是供不同的域名(服务)使用的，不同的服务的证书可能会不相同，此时这样我们可以使用traefik 给大家提供的统一的 https 证书，当然我们也可以单独为当前的服务提供单独的证书，同样用证书文件创建一个 secret 对象，然后在 ingress 对象中声明一个 tls 对象即可，比如上面的 example.haimaxy.com 我们可以单独指定一个证书文件：

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: example-web-app
  annotations:
    kubernetes.io/ingress.class: "traefik"
spec:
  tls:
    - secretName: traefik-cert
  rules:
  - host:

参考

官方文档：https://docs.traefik.io/user-guide/kubernetes/