Using kubeadm to install kubernetes cluster is very convenient, but there is also an annoying problem that the default certificate validity period is only one year, so you need to consider the issue of certificate upgrade. The demo cluster version in this article is v1.17.0 version, and the following operations are not guaranteed. Other versions are also applicable, and the certificate directory must be backed up before operation to prevent operation errors from being rolled back. This article mainly introduces two ways to update the cluster certificate.
##Manually renew the certificate 
This command displays the client certificate in the /etc/kubernetes/pki folder and the expiration time/remaining time of the client certificate embedded in the KUBECONFIG file used by kubeadm.
Kubeadm cannot manage certificates signed by external CAs. If it is an external certificate, you need to manually manage the renewal of the certificate.
Another thing to note is that kubelet.conf is not included in the above list, because kubeadm configures kubelet to automatically update the certificate.
In addition, kubeadm will automatically update all certificates when the control panel is upgraded, so the best way to build a cluster with kubeadm is to upgrade the cluster frequently, so as to ensure that your cluster stays up to date and maintains reasonable security. But for the actual production environment, we may not upgrade the cluster frequently, so at this time we need to manually update the certificate
It is also very convenient to manually update the certificate, we only need to update your certificate through the kubeadm alpha certs renew command, this command uses the CA (or front-proxy-CA) certificate and the key stored in /etc/kubernetes/pki Perform the update.
If you are running a highly available cluster, this command needs to be executed on all control panel nodes.
Next we will update our cluster certificate. The following operations are all performed on the master node. First, back up the original certificate:
[root@k8s-1 ~]# mkdir /etc/kubernetes.bak
[root@k8s-1 ~]# cp -r /etc/kubernetes/pki/ /etc/kubernetes.bak/
[root@k8s- 1 ~]#cp /etc/kubernetes/*.conf /etc/kubernetes.bak/
Then back up the etcd data directory:
[root@k8s-1 ~]# cp -r /var/lib/etcd /var/lib/etcd.bak
Next execute the command to update the certificate:
[root@k8s-1 ~]# kubeadm config view> kubeadm.yaml If
there is no kubeadm.yaml file, first generate the yaml file
and then update the certificate
The one-click update of the certificate is completed through the above command. At this time, you can see that the expiration time is one year later by viewing the above certificate:
Then remember to update the kubeconfig file:
Overwrite the original admin file with the newly generated admin configuration file:
[root@k8s-1 ~]# mv $HOME/.kube/config $HOME/.kube/config.old
[root@k8s-1 ~]# cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
[root@k8s-1 ~]# chown $(id -u):$(id -g) $HOME/.kube/config
After completion, restart the four containers kube-apiserver, kube-controller, kube-scheduler, etcd. We can check the validity period of the apiserver certificate to verify whether the update is successful:
[root@k8s-1 ~]# echo | openssl s_client -showcerts -connect 127.0.0.1:6443 -servername api 2>/dev/null | openssl x509 -noout -enddate
notAfter=Oct 21 07:19:04 2021 GMT
You can see that the current validity period is one year later, which proves that the update has been successful.
##Update the certificate with the Kubernetes certificate API
In addition to the one-click manual certificate update mentioned above, you can also use the Kubernetes certificate API to perform manual certificate update. For the online environment, we may not take the risk of frequently updating the cluster or updating the certificate. These are risky after all. Therefore, we hope that the validity period of the generated certificate is long enough. Although this is not recommended from a security perspective, it is not recommended. In some scenarios, a sufficiently long certificate validity period is also very necessary. Many administrators manually change the source code of kubeadm to 10 years, and then recompile to create a cluster. Although this method can achieve the purpose, it is not recommended to use this method, especially when you want to update the cluster. It must be updated with the new version. In fact, Kubernetes provides an API method to help us generate a long enough certificate validity period.
To use the built-in API method to sign, first we need to configure the --experimental-cluster-signing-duration parameter of the kube-controller-manager component and adjust it to 10 years. Here is the cluster installed by kubeadm, so we directly modify the static The Pod's yaml file can be:
vim /etc/kubernetes/manifests/kube-controller-manager.yaml
After the modification is completed, kube-controller-manager will automatically restart to take effect. Then we need to use the following command to create a certificate signing request for the Kubernetes certificate API. If you set up an external signer such as cert-manager, certificate signing requests (CSRs) will be automatically approved. Otherwise, you must manually approve the certificate using the kubectl certificate command. The following kubeadm command outputs the name of the certificate to be approved, and then waits for approval to occur:
$ kubeadm alpha certs renew all --use-api --config kubeadm.yaml &
The output is similar to the following:
Then we need to manually approve the certificate:
[root @ k8s-1 ~] # kubectl get csr
NAME AGE REQUESTOR CONDITION
kubeadm-cert-kube-apiserver-dcc55 88s kubernetes-admin Pending
kubeadm-cert-kubernetes-admin-r54fc 5m18s kubernetes-admin Pending
kubeadm-cert admin-x4qjq 2m52s kubernetes-admin Approved, Issued
Manually approve the certificate
$ kubectl certificate approve kubeadm-cert-kubernetes-admin-r54fc
certificatesigningrequest.certificates.k8s.io/kubeadm-cert-kubernetes-admin-pn99f approved
Every time a csr is updated, a new csr will be generated
Perform the approval operation for the csr in the Pending state in the same way until all csrs are approved. Finally, the status of all csr lists is as follows:
Check the validity period of the certificate after approval:
We can see that it has been extended for 10 years. This is because the validity period of the CA certificate is only 10 years.
But now we can't directly restart several components of the control panel. This is because the etcd corresponding to the cluster installed with kubeadm uses the /etc/kubernetes/pki/etcd/ca.crt certificate to perform the previous step by default. The certificate approved by the command kubectl certificate approve is issued with the default /etc/kubernetes/pki/ca.crt certificate, so we need to replace the CA certificate in etcd:
##First copy the static Pod resource list
[root@k8s-1 ~]# cp -r /etc/kubernetes/manifests/ /etc/kubernetes/manifests.bak
[root@k8s-1 ~]# vim /etc/kubernetes/manifests/etcd.yaml
Since kube-apiserver needs to connect to the etcd cluster, the corresponding etcd ca file also needs to be revised:
[root@k8s-1 ~]# vim /etc/kubernetes/manifests/kube-apiserver.yaml
In addition, you need to replace the requestheader-client-ca-file file, the default is the /etc/kubernetes/pki/front-proxy-ca.crt file, and now you need to replace it with the default CA file, otherwise use the aggregation API, such as After installing the metrics-server and executing the kubectl top command, an error will be reported:
[root@k8s-1 ~]# cp /etc/kubernetes/pki/ca.crt /etc/kubernetes/pki/front-proxy-ca.crt
[root@k8s-1 ~]# cp /etc/kubernetes/pki/ca.key /etc/kubernetes/pki/front-proxy-ca.key
Since it is a static Pod, the above components will automatically restart to take effect after the modification is completed. Since our current version of kubelet has automatic certificate rotation enabled by default, the kubelet certificate does not need to be managed anymore, so I will update the certificate to a validity period of 10. Be sure to back up the certificate directory before operation to prevent rollback of operation errors.