轻量级K8S-K3S上手指南

K3S是轻量级的K8S，这点从名字上就能看出来。适用于边缘计算、物联网、CI、ARM。

一、K3S的优点

完美适合边缘计算场景

K3S是一个高可用、经过认证的Kubernetes发行版，专为无人值守、资源受限、远端或物联网设备内部的生产负载而设计。

简单安全

K3S是一个小于40MB的二进制文件包，减少了安装、运行和自动更新一个生产Kubernetes集群所需的依赖性和步骤。

针对ARM优化

支持ARM64和ARMv7，提供二进制文件和多架构镜像。从树莓派到AWS a1.4XL 32GiB服务器，K3S都能运行良好。

二、工作原理

三、快速部署

脚本安装

1. 安装文件准备

下载 K3S最新版本, 目前x86_64, ARMv7, 和ARM64均支持。

2. 启动服务

sudo k3s server &
# Kubeconfig被写入到/etc/rancher/k3s/k3s.yaml，且服务会自动启动或重启。
# 安装脚本会自动安装K3S和kubectl等工
sudo k3s kubectl get node

# 在其他节点跑以下命令。NODE_TOKEN来自/var/lib/rancher/k3s/server/node-token
# 在服务器上执行
sudo k3s agent --server https://myserver:6443 --token ${NODE_TOKEN}

容器部署

1. 环境准备

以centos 7 为例

更新yum源

sudo wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo
yum makecache fast

2. 安装docker

1. 安装前置依赖

sudo yum install -y yum-utils \
  device-mapper-persistent-data \
  lvm2

2. 添加yum源

yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
yum makecache fast

3. 查看可安装列表

yum list docker-ce --showduplicates | sort -r

4. 安装最新版(默认)

sudo yum install docker-ce docker-ce-cli containerd.io

5. 启动

systemctl enable docker
systemctl start docker

3. 安装docker-compose

sudo curl -L "https://github.com/docker/compose/releases/download/1.25.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

sudo chmod +x /usr/local/bin/docker-compose

docker-compose -v

4. 下载kubectl

curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl

mv kubectl /usr/local/bin/kubectl

chmod +x /usr/local/bin/kubectl

5. 编写k3s docker-compose文件

version: '3'
services:
  server:
    image: rancher/k3s:v0.8.0-amd64
    command: server --disable-agent
    environment:
    - K3S_CLUSTER_SECRET=somethingtotallyrandom
    - K3S_KUBECONFIG_OUTPUT=/output/kubeconfig.yaml
    - K3S_KUBECONFIG_MODE=666
    volumes:
    - k3s-server:/var/lib/rancher/k3s
    # This is just so that we get the kubeconfig file out
    - .:/output
    ports:
    - 6443:6443

  node:
    image: rancher/k3s:v0.8.0-amd64
    tmpfs:
    - /run
    - /var/run
    privileged: true
    command: agent --pause-image=gcr.azk8s.cn/google_containers/pause:3.1 --kubelet-arg pod-infra-container-image=gcr.azk8s.cn/google_containers/pause:3.1
    depends_on:
    - server
    environment:
    - K3S_URL=https://server:6443
    - K3S_CLUSTER_SECRET=somethingtotallyrandom
    # Can also use K3S_TOKEN from /var/lib/rancher/k3s/server/node-token instead of K3S_CLUSTER_SECRET
    #- K3S_TOKEN=K13849a67fc385fd3c0fa6133a8649d9e717b0258b3b09c87ffc33dae362c12d8c0::node:2e373dca319a0525745fd8b3d8120d9c

volumes:
  k3s-server: {}

6. 启动

docker-compose up -d
docker-compose ps

7. 配置

将生产的配置文件拷贝到~/.kube目录下

mkdir ~/.kube
cp kubeconfig.yaml ~/.kube/config
kubectl get node
kubectl get po --all-namespaces

8. 示例

mynginx.yml

---
apiVersion: apps/v1
kind: Deployment
metadata:
  generation: 1
  labels:
    run: mynginx
  name: mynginx
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      run: mynginx
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      creationTimestamp: null
      labels:
        run: mynginx
    spec:
      containers:
      - image: registry.xxx.com.cn/tools/nginx:1.11.4-alpine
        imagePullPolicy: IfNotPresent
        name: mynginx
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30
---
apiVersion: v1
kind: Service
metadata:
  labels:
    run: mynginx
  name: mynginx
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: mynginx
  sessionAffinity: None
  type: ClusterIP
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: mynginx
spec:
  rules:
  - host: mynginx.abc.com
    http:
      paths:
      - backend:
          serviceName: mynginx
          servicePort: 80

docker-compose.yml

version: '3'
services:
  server:
    image: rancher/k3s:v0.8.0-amd64
    command: server --disable-agent
    environment:
    - K3S_CLUSTER_SECRET=somethingtotallyrandom
    - K3S_KUBECONFIG_OUTPUT=/output/kubeconfig.yaml
    - K3S_KUBECONFIG_MODE=666
    volumes:
    - k3s-server:/var/lib/rancher/k3s
    # This is just so that we get the kubeconfig file out
    - .:/output
    ports:
    - 6443:6443

  node:
    image: rancher/k3s:v0.8.0-amd64
    tmpfs:
    - /run
    - /var/run
    privileged: true
    command: agent --pause-image=gcr.azk8s.cn/google_containers/pause:3.1 --kubelet-arg pod-infra-container-image=gcr.azk8s.cn/google_containers/pause:3.1
    depends_on:
    - server
    environment:
    - K3S_URL=https://server:6443
    - K3S_CLUSTER_SECRET=somethingtotallyrandom
    # Can also use K3S_TOKEN from /var/lib/rancher/k3s/server/node-token instead of K3S_CLUSTER_SECRET
    #- K3S_TOKEN=K13849a67fc385fd3c0fa6133a8649d9e717b0258b3b09c87ffc33dae362c12d8c0::node:2e373dca319a0525745fd8b3d8120d9c

volumes:
  k3s-server: {}

# 创建命名空间
kubectl create ns tools
# 应用编排文件
kubectl apply -f mynginx.yml -n tools
# 查看创建的资源
[root@k3s ~]# kubectl get  po -n tools
NAME                       READY   STATUS    RESTARTS   AGE
mynginx-6b9d6d4579-fglzv   1/1     Running   0          27m
[root@k3s ~]# kubectl get svc -n tools
NAME      TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
mynginx   ClusterIP   10.43.104.241   <none>        80/TCP    27m
[root@k3s ~]# kubectl get ing -n tools
NAME      HOSTS             ADDRESS      PORTS   AGE
mynginx   mynginx.abc.com   172.18.0.3   80      25m

# 测试访问
echo "172.18.0.3 mynginx.abc.com" >> /etc/hosts
curl http://mynginx.abc.com

9.番外篇

由于虚机中启动k3s，如果想要主机访问到k3s的域名，可在虚机中做DNAT转发，如下，只需将主机80端口转发至172.18.0.3:80

[root@k3s ~]# kubectl get svc -n kube-system
NAME       TYPE           CLUSTER-IP   EXTERNAL-IP   PORT(S)                      AGE
kube-dns   ClusterIP      10.43.0.10   <none>        53/UDP,53/TCP,9153/TCP       154m
traefik    LoadBalancer   10.43.9.36   172.18.0.3    80:30033/TCP,443:30849/TCP   152m

添加如下规则：

echo "net.ipv4.ip_forward=1" >> /etc/sysctl.conf
sysctl -p /etc/sysctl.conf
iptables -t filter -A FORWARD -j ACCEPT
iptables -t nat -A PREROUTING -p tcp -m tcp --dport 80 -j DNAT --to-destination 172.18.0.3:80
iptables -t nat -A PREROUTING -p tcp -m tcp --dport 443 -j DNAT --to-destination 172.18.0.3:443

宿主机添加hosts，浏览器打开网站访问mynginx.abc.com:

echo "[虚机IP] mynginx.abc.com" >> /etc/hosts

参考资料

官网：https://k3s.io
github：https://github.com/rancher/k3s