主节点更改

KUBE-APISERVER

添加--enable-bootstrap-token-auth=true、--client-ca-file=FILENAME到您的kube-apiserver服务文件。最后，您的服务文件应如下所示：

cat <<EOF | sudo tee /etc/systemd/system/kube-apiserver.service
[Unit]
Description=Kubernetes API Server
Documentation=https://github.com/kubernetes/kubernetes

[Service]
ExecStart=/usr/local/bin/kube-apiserver \\
  --advertise-address=${INTERNAL_IP} \\
  --allow-privileged=true \\
  --apiserver-count=3 \\
  --audit-log-maxage=30 \\
  --audit-log-maxbackup=3 \\
  --audit-log-maxsize=100 \\
  --audit-log-path=/var/log/audit.log \\
  --authorization-mode=Node,RBAC \\
  --bind-address=0.0.0.0 \\
  --client-ca-file=/var/lib/kubernetes/ca.pem \\
  --enable-admission-plugins=Initializers,NamespaceLifecycle,NodeRestriction,LimitRanger,ServiceAccount,DefaultStorageClass,ResourceQuota \\
  --enable-bootstrap-token-auth=true \\
  --enable-swagger-ui=true \\
  --etcd-cafile=/var/lib/kubernetes/ca.pem \\
  --etcd-certfile=/var/lib/kubernetes/kubernetes.pem \\
  --etcd-keyfile=/var/lib/kubernetes/kubernetes-key.pem \\
  --etcd-servers=https://192.168.50.10:2379 \\
  --event-ttl=1h \\
  --experimental-encryption-provider-config=/var/lib/kubernetes/encryption-config.yaml \\
  --kubelet-certificate-authority=/var/lib/kubernetes/ca.pem \\
  --kubelet-client-certificate=/var/lib/kubernetes/kubernetes.pem \\
  --kubelet-client-key=/var/lib/kubernetes/kubernetes-key.pem \\
  --kubelet-https=true \\
  --runtime-config=api/all \\
  --service-account-key-file=/var/lib/kubernetes/service-account.pem \\
  --service-cluster-ip-range=10.32.0.0/24 \\
  --service-node-port-range=30000-32767 \\
  --tls-cert-file=/var/lib/kubernetes/kubernetes.pem \\
  --tls-private-key-file=/var/lib/kubernetes/kubernetes-key.pem \\
  --v=2
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target
EOF

KUBE-CONTROLLER-MANAGER

添加--cluster-signing-cert-file=/var/lib/kubernetes/ca.pem 和 --cluster-signing-key-file=/var/lib/kubernetes/ca-key.pem以及 --controllers=*,bootstrapsigner,tokencleaner到kube-controller-manager.service文件中。因此服务文件应如下所示：

cat <<EOF | sudo tee /etc/systemd/system/kube-controller-manager.service
[Unit]
Description=Kubernetes Controller Manager
Documentation=https://github.com/kubernetes/kubernetes

[Service]
ExecStart=/usr/local/bin/kube-controller-manager \\
  --address=0.0.0.0 \\
  --cluster-cidr=10.200.0.0/16 \\
  --cluster-name=kubernetes \\
  --cluster-signing-cert-file=/var/lib/kubernetes/ca.pem \\
  --cluster-signing-key-file=/var/lib/kubernetes/ca-key.pem \\
  --controllers=*,bootstrapsigner,tokencleaner \\
  --kubeconfig=/var/lib/kubernetes/kube-controller-manager.kubeconfig \\
  --leader-elect=true \\
  --root-ca-file=/var/lib/kubernetes/ca.pem \\
  --service-account-private-key-file=/var/lib/kubernetes/service-account-key.pem \\
  --service-cluster-ip-range=10.32.0.0/24 \\
  --use-service-account-credentials=true \\
  --v=2
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target
EOF

BOOTSTRAP SECRET

我们用来设置的引导令牌是07401b.f395accd246ae52d（您可以自己生成一个）。该令牌有两个部分，分别是公共ID07401b和私有部分f395accd246ae52d。在此处详细了解令牌。还有关于 bootstrap token secret 格式的信息。

可以通过以下方式生成 BOOTSTRAP Token

按照TLS Bootstrapping的文档，我们首先创建 token 认证文件。创建一个具有以下内容的文件：

$ cat tokenfile 
02b50b05283e98dd0fd71db496ef01e8,kubelet-bootstrap,10001,"system:bootstrappers"

您应该通过运行以下命令来创建尽可能随机的 token ：

head -c 16 /dev/urandom | od -An -t x | tr -d ' '

现在我们需要告诉kube apiserver有关此文件的信息。因此，将以下标志添加到kube-apiserver服务文件中，并带有上标记文件的路径。

--token-auth-file=/home/vagrant/tokenfile

创建以下 secret 要具有一定要求。 secret 名称应采用如下格式： format bootstrap-token-<token public id> 并且应在kube-system名称空间中可用。

cat <<EOF | kubectl apply --kubeconfig admin.kubeconfig -f -
apiVersion: v1
kind: Secret
metadata:
  # Name MUST be of form "bootstrap-token-<token id>"
  name: bootstrap-token-07401b
  namespace: kube-system

# Type MUST be 'bootstrap.kubernetes.io/token'
type: bootstrap.kubernetes.io/token
stringData:
  # Human readable description. Optional.
  description: "Created for Kubernetes the Hard Way"

  # Token ID and secret. Required.
  token-id: 07401b
  token-secret: f395accd246ae52d

  # Allowed usages.
  usage-bootstrap-authentication: "true"
  usage-bootstrap-signing: "true"
EOF

启用BOOTSTRAPPING的RBAC策略

通过该令牌认证的用户属于该组system:bootstrappers，这就是为什么向该组授予以下权限。

kubectl create clusterrolebinding kubelet-bootstrap \
  --clusterrole=system:node-bootstrapper \
  --group=system:bootstrappers

kubectl create clusterrolebinding node-autoapprove-bootstrap \
  --clusterrole=system:certificates.k8s.io:certificatesigningrequests:nodeclient \
  --group=system:bootstrappers

kubectl create clusterrolebinding node-autoapprove-certificate-rotation \
  --clusterrole=system:certificates.k8s.io:certificatesigningrequests:selfnodeclient \
  --group=system:nodes

工作节点更改

KUBELET

添加--bootstrap-kubeconfig，这是kubeconfig的路径，它包含 bootstrap token 和与kube-apiserver的交互信息。同样，您无需提供--kubeconfig，而是提供路径，该路径将自动生成并保存在该路径中。

您的kubelet服务文件应如下所示：

cat <<EOF | sudo tee /etc/systemd/system/kubelet.service
[Unit]
Description=Kubernetes Kubelet
Documentation=https://github.com/kubernetes/kubernetes
After=containerd.service
Requires=containerd.service

[Service]
ExecStart=/usr/local/bin/kubelet \\
  --bootstrap-kubeconfig=/home/vagrant/bootstrap.kubeconfig \\
  --config=/var/lib/kubelet/kubelet-config.yaml \\
  --container-runtime=remote \\
  --container-runtime-endpoint=unix:///var/run/containerd/containerd.sock \\
  --image-pull-progress-deadline=2m \\
  --kubeconfig=/home/vagrant/kubeconfig \\
  --network-plugin=cni \\
  --register-node=true \\
  --v=2
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target
EOF

现在更改kubelet配置文件

cat <<EOF | sudo tee /var/lib/kubelet/kubelet-config.yaml
kind: KubeletConfiguration
apiVersion: kubelet.config.k8s.io/v1beta1
authentication:
  anonymous:
    enabled: false
  webhook:
    enabled: true
  x509:
    clientCAFile: "/home/vagrant/ca.pem"
authorization:
  mode: Webhook
clusterDomain: "cluster.local"
clusterDNS:
  - "10.32.0.10"
podCIDR: "${POD_CIDR}"
rotateCertificates: true
runtimeRequestTimeout: "15m"
serverTLSBootstrap: true
EOF

提供clientCAFile适当的路径。并添加另外两个字段rotateCertificates: true＆serverTLSBootstrap: true，这将启用证书轮换。

创建BOOTSTRAP KUBECONFIG

# I have used the ip address of my kube kube-apiserver use yours
kubectl config set-cluster kthwkinvolk \
  --certificate-authority=ca.pem \
  --embed-certs=true \
  --server=https://192.168.50.10:6443 \
  --kubeconfig=/home/vagrant/bootstrap.kubeconfig

# this token is above generated
kubectl config set-credentials kubelet-bootstrap \
  --token=07401b.f395accd246ae52d \
  --kubeconfig=/home/vagrant/bootstrap.kubeconfig

kubectl config set-context default \
  --cluster=kthwkinvolk \
  --user=kubelet-bootstrap \
  --kubeconfig=/home/vagrant/bootstrap.kubeconfig

kubectl config use-context default \
  --kubeconfig=/home/vagrant/bootstrap.kubeconfig