Gitlab+Rancher realizes Dockerized DevOps Part II: CD to K8S cluster <<Unfinished>>

product comparison

• Docker solves single-machine deployment and single-container
• Compose solves the problem of deploying multiple containers on a single machine
• Swarm solves multi-machine deployment and multi-container (lightweight)
• K8S powerful cluster deployment solution

Building a K8S cluster based on Rancher

• The V1 version supports K8S, Mesos, Swarm, and V2 fully supports the only K8S
• Catalog: An app marketplace built by rancher
• Cattle: the orchestration and scheduling framework used by rancher itself

Install
firewall open ports

• SSH：22/tcp
• RancherServer：8443/tcp、8080/tcp
• K8S ：6443(tcp ApiServer)、10250(tcp KubeletApi)、10251(tcp Schedule)、10252(tcp Control)、10255(tcp Control)、10256(tcp Kubeproxy)、30000/32767(tcp NodePort)
• VXLAN：4789/udp
• IPSec：500/udp、4500/udp
• Etcd：2379/tcp、2380/tcp
• Canal：80/tcp、443/tcp
• Flannel：8285/udp、8472/udp、2375/udp

Enable IPV4 routing forwarding (CentOS 7.4+ does not have to do this)

#/etc/sysctl.conf追加一行
net.ipv4.ip_forward = 1

Docker ready

• RancherServer and cluster nodes support the highest Docker versionv17.03-ce
• sudo yum install -y --setopt=obsoletes=0 docker-ce-17.03.2.ce-1.el7.centos docker-ce-selinux-17.03.2.ce-1.el7.centos

HTTPS certificate preparation

docker run -it --rm -p 443:443 -p 80:80 --name certbot \
            -v "/etc/letsencrypt:/etc/letsencrypt" \
            -v "/var/lib/letsencrypt:/var/lib/letsencrypt" \
            certbot/certbot certonly -n -v --standalone --agree-tos [email protected] -d rancher.example.com

cd /etc/letsencrypt
sudo ln -s live/rancher.example.com/fullchain.pem cert.pem
sudo ln -s live/rancher.example.com/privkey.pem key.pem

Node machine adjustment

• Creating an RKE cluster in a custom way requires the node hostname

# hostname要求符合如下正则 `'[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*'`
sudo hostnamectl set-hostname k8s-worker-1.cluster-a
sudo hostnamectl status

• In order to speed up the speed of the node and pull the private mirror library, it is necessary to add a host to the local area network ip of the mirror library to resolve the mirror library on the node machine

Compose Orchestration Service

version: '2' 

services:

  Rancher:
    image: rancher/server:preview
    container_name: rancher
    hostname: rancher
    restart: always
    ports:
      - '8443:8443'
      - '8080:8080'
    volumes:
        - /srv/rancher:/var/lib/rancher
        - /etc/letsencrypt:/etc/rancher/ssl
    entrypoint: rancher --http-listen-port=8080 --https-listen-port=8443
    command: --acme-domain rancher.example.com

start the service

docker pull rancher/server:preview
docker-compose up -d Rancher
docker logs -f rancher #跟进rancher初始化状态

configure

• Default account passwordadmin:admin
• Log in to the system and change the password

Create a cluster

• custom mode, canal network to create a cluster
• The control and etcd nodes require at least 1 core and 2G memory (the cluster node is offline to check the machine load)
• Configure Registries Private Mirror Repository

Cluster tuning

• Forbid non-worker nodes to schedule running PODskubectl taint node 节点名 node-role.kubernetes.io/节点名="":NoSchedule

debugging

####
#RancherServer调试
docker logs -f rancher

#K8sNode调试
journalctl -xf -u docker
docker logs kubelet

Deploy Traefik agent

Uninstall rancher pre-installed nginx-ingress-controler

• mainly includessrv default-http-backend、deploy default-http-backend、ds nginx-ingress-controller、sa nginx-ingress-serviceaccount、clusterrolebindings nginx-ingress-clusterrole-nisa-binding、
• implementkubectl delete namespace ingress-nginx

RBAC authorization

---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
rules:
  - apiGroups:
      - ""
    resources:
      - services
      - endpoints
      - secrets
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: traefik-ingress-controller
subjects:
  - kind: ServiceAccount
    name: traefik-ingress-controller
    namespace: kube-system

DaemonSet deployment

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: traefik-ingress-controller
  namespace: kube-system
---
kind: DaemonSet
apiVersion: extensions/v1beta1
metadata:
  name: traefik-ingress-controller
  namespace: kube-system
  labels:
    k8s-app: traefik-ingress-lb
spec:
  template:
    metadata:
      labels:
        k8s-app: traefik-ingress-lb
        name: traefik-ingress-lb
    spec:
      serviceAccountName: traefik-ingress-controller
      terminationGracePeriodSeconds: 60
      hostNetwork: true
      containers:
      - image: traefik
        name: traefik-ingress-lb
        ports:
        - name: http
          containerPort: 80
          hostPort: 80
        - name: admin
          containerPort: 8080
        securityContext:
          privileged: true
        args:
        - --api
        - --kubernetes
        - --logLevel=INFO
---
kind: Service
apiVersion: v1
metadata:
  name: traefik-ingress-service
  namespace: kube-system
spec:
  selector:
    k8s-app: traefik-ingress-lb
  ports:
    - protocol: TCP
      port: 80
      name: web
    - protocol: TCP
      port: 8080
      name: admin
  type: NodePort

Create TraefikUI entry

apiVersion: v1
kind: Service
metadata:
  name: traefik-web-ui
  namespace: kube-system
spec:
  selector:
    k8s-app: traefik-ingress-lb
  ports:
  - port: 80
    targetPort: 8080
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: traefik-web-ui
  namespace: kube-system
  annotations:
    kubernetes.io/ingress.class: traefik
spec:
  rules:
  - host: traefik-ui.example.com
    http:
      paths:
      - backend:
          serviceName: traefik-web-ui
          servicePort: 80

CD deployment application

{{o.name}}

{{m.name}}