Centos7 system uses kubeadm to install k8s cluster v1.26.1

Install k8s cluster in kubeadm mode

1. Prepare the machine

host	illustrate
192.168.0.11	The master node can connect to the external network, and the official website requires at least 2 cores and 2G
192.168.0.12	node1 node, can connect to the external network, the official website requires at least 2 cores 2G
192.168.0.13	node2 node, can connect to the external network, the official website requires at least 2 cores 2G

2. Server environment configuration

2.1 Turn off the firewall (all nodes)

Turn off the firewall and set the boot to not start

systemctl stop firewalld
systemctl disable firewalld

2.2 Disable selinux (all nodes)

#修改/etc/selinux/config文件中的SELINUX=disabled
vim /etc/selinux/config

2.3 Close the swap partition (all nodes)

Restart the server after modification to take effect

vim /etc/fstab						#永久禁用swap,删除或注释掉/etc/fstab里的swap设备的挂载命令即可
#/dev/mapper/centos-swap swap                    swap    defaults        0 0

2.4 Centos7 kernel upgrade (all nodes)

There are some bugs in the 3.10.x kernel that comes with the CentOS 7.x system, which makes running Docker and Kubernetes unstable.
Reference: Upgrading the centos system kernel

2.5 Set the hostname (all nodes)

cat >> /etc/hosts <<EOF
192.168.0.11 master
192.168.0.12 worker01
192.168.0.13 worker02
EOF

2.6 Time synchronization (all nodes)

yum -y install ntp
systemctl start ntpd
systemctl enable ntpd

3. Install docker (all nodes)

Follow this method to install docker : centos install docker-ce

configure docker

#registry-mirrors是配置docker镜像源
#exec-opts是配置Cgroup Driver为systemd，因为k8s使用的是systemd
cat <<EOF > /etc/docker/daemon.json
{
    "registry-mirrors": [
        "http://hub-mirror.c.163.com",
        "https://docker.mirrors.ustc.edu.cn",
        "https://registry.docker-cn.com"
    ],
    "exec-opts": ["native.cgroupdriver=systemd"]
}
EOF

#重启docker后查看是否生效
systemctl restart docker
docker info | grep -i "Cgroup Driver"

4. Install cri-dockerd (all nodes)

#从https://github.com/Mirantis/cri-dockerd/releases中下载最新的rpm包,手动下载后上传到服务器里
rpm -ivh cri-dockerd-0.3.1-3.el7.x86_64.rpm

#修改/usr/lib/systemd/system/cri-docker.service文件中的ExecStart配置
vim /usr/lib/systemd/system/cri-docker.service
ExecStart=/usr/bin/cri-dockerd --network-plugin=cni --pod-infra-container-image=registry.aliyuncs.com/google_containers/pause:3.7

systemctl daemon-reload
systemctl enable --now cri-docker

5. Configure the Alibaba Cloud yum source of kubernetes (all nodes)

The x86_64 value at the end of the baseurl address needs to be modified according to the system, enter uname -m to view the value. For example, the baseurl URL for x86_64 could be: https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64

cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

Six, yum install kubeadm, kubelet, kubectl (all nodes)

Install kubeadm, kubelet, and kubectl on all three virtual machines (kubeadm and kubectl are tools, and kubelet is a system service)

#删除之前的
yum -y remove kubelet kubeadm kubectl

#查看yum可获取的kubeadm版本，这里安装1.26.1版本，不指定版本的话默认安装最新版本
yum list --showduplicates | grep  kubeadm
#安装kubeadm、kubelet、kubectl
yum -y install kubelet-1.26.1 kubeadm-1.26.1 kubectl-1.26.1
#设置kubelet开机自启（先不用启动，也起不了，后面kubeadm init初始化master时会自动拉起kubelet）
systemctl enable kubelet

7. Initialize the control panel of the master node (master node)

Executing the following command may report an error, and modify it accordingly according to this Kubernetes common error report

# kubeadm init --help可以查看命令的具体参数用法

#在master节点执行初始化（node节点不用执行）
#apiserver-advertise-address  指定apiserver的IP，即master节点的IP
#image-repository  设置镜像仓库为国内的阿里云镜像仓库
#kubernetes-version  设置k8s的版本，跟步骤三的kubeadm版本一致
#service-cidr  这是设置node节点的网络的，暂时这样设置
#pod-network-cidr  这是设置node节点的网络的，暂时这样设置
#cri-socket  设置cri使用cri-dockerd

kubeadm init \
--apiserver-advertise-address=192.168.0.11 \
--image-repository registry.aliyuncs.com/google_containers \
--kubernetes-version v1.26.1 \
--service-cidr=10.96.0.0/12 \
--pod-network-cidr=10.244.0.0/16 \
--cri-socket unix:///var/run/cri-dockerd.sock \
--ignore-preflight-errors=all

If there is an error in the above kubeadm init command, execute the following command to reset kubeadm and useless images

#重置Kubeadm
kubeadm reset -f

#删除docker无用的镜像容器
docker system prune -f

After the above execution, the following results appear to indicate success

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.0.11:6443 --token xw8o4d.ly5o9kxgbodtrykw \
        --discovery-token-ca-cert-hash sha256:2fbb2be8829dd90f789b13269f2ef4d8de6a39bc568c61e3a6a00ea3c95efd94

Execute the corresponding command on the master node according to the above result (just copy the command prompted above)

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
export KUBECONFIG=/etc/kubernetes/admin.conf

8. Add the node node to the k8s cluster

#命令最后加上cri-socket指定使用cri-dockerd
kubeadm join 192.168.0.11:6443 --token xw8o4d.ly5o9kxgbodtrykw \
        --discovery-token-ca-cert-hash sha256:2fbb2be8829dd90f789b13269f2ef4d8de6a39bc568c61e3a6a00ea3c95efd94 \
        --cri-socket unix:///var/run/cri-dockerd.sock

After the above command reports an error, make corresponding modifications according to the content of the error, for example, the following error message

[preflight] Running pre-flight checks
        [WARNING Swap]: swap is enabled; production deployments should disable swap unless testing the NodeSwap feature gate of the kubelet
error execution phase preflight: [preflight] Some fatal errors occurred:
        [ERROR FileAvailable--etc-kubernetes-kubelet.conf]: /etc/kubernetes/kubelet.conf already exists
        [ERROR Port-10250]: Port 10250 is in use
        [ERROR FileAvailable--etc-kubernetes-pki-ca.crt]: /etc/kubernetes/pki/ca.crt already exists
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher

Delete the existing configuration file and restart kubelet

kubeadm reset --cri-socket unix:///var/run/cri-dockerd.sock


rm -f /etc/kubernetes/kubelet.conf
rm -f /etc/kubernetes/pki/ca.crt
systemctl stop kubelet

9. Configure pod network creation on the master node

After executing kubectl get nodes, it is found that they are all NotReady, and the CNI network plug-in needs to be configured

[root@master ~]# kubectl get nodes
NAME       STATUS     ROLES           AGE   VERSION
master     NotReady   control-plane   58m   v1.26.1
worker01   NotReady   <none>          37m   v1.26.1
worker02   NotReady   <none>          25m   v1.26.1

#在master节点配置pod网络创建
#node节点加入k8s集群后，在master上执行kubectl get nodes发现状态是NotReady，因为还没有部署CNI网络插件，其实在步骤四初始化
#完成master节点的时候k8s已经叫我们去配置pod网络了。在k8s系统上Pod网络的实现依赖于第三方插件进行，这类插件有近数十种之多，较为
#著名的有flannel、calico、canal和kube-router等，简单易用的实现是为CoreOS提供的flannel项目。

#执行下面这条命令在线配置pod网络，因为是国外网站，所以可能报错，测试去http://ip.tool.chinaz.com/网站查到
#域名raw.githubusercontent.com对应的IP，把域名解析配置到/etc/hosts文件，然后执行在线配置pod网络，多尝试几次即可成功。
[root@master ~]# kubectl apply -f https://raw.githubusercontent.com/flannel-io/flannel/master/Documentation/kube-flannel.yml					
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy/psp.flannel.unprivileged created
clusterrole.rbac.authorization.k8s.io/flannel created
clusterrolebinding.rbac.authorization.k8s.io/flannel created
serviceaccount/flannel created
configmap/kube-flannel-cfg created
daemonset.apps/kube-flannel-ds created
[root@master ~]# kubectl get pods -n kube-system						#查看pod状态
NAME                             READY   STATUS     RESTARTS   AGE
coredns-7f6cbbb7b8-bm2gl         0/1     Pending    0          86m
coredns-7f6cbbb7b8-frq8l         0/1     Pending    0          86m
etcd-master                      1/1     Running    1          87m
kube-apiserver-master            1/1     Running    1          87m
kube-controller-manager-master   1/1     Running    1          87m
kube-flannel-ds-5rwkt            0/1     Init:1/2   0          2m13s
kube-flannel-ds-9fqkl            1/1     Running    0          2m13s
kube-flannel-ds-bvgh4            1/1     Running    0          2m13s
kube-proxy-8vmqg                 1/1     Running    0          59m
kube-proxy-ll9hw                 1/1     Running    0          86m
kube-proxy-zndg7                 1/1     Running    0          59m
kube-scheduler-master            1/1     Running    1          87m
# 重启服务器后获取
[root@master ~]# kubectl get nodes										#pod网络已经配置完成，状态已经是Ready
NAME       STATUS     ROLES           AGE   VERSION
master     Ready      control-plane   58m   v1.26.1
worker01   Ready      <none>          37m   v1.26.1
worker02   Ready      <none>          25m   v1.26.1

10. Test the k8s cluster

Create a pod in k8s and verify that it is running normally

[root@master ~]# kubectl create deployment httpd --image=httpd					#创建一个httpd服务测试
deployment.apps/httpd created
[root@master ~]# kubectl expose deployment httpd --port=80 --type=NodePort		#端口就写80,如果你写其他的可能防火墙拦截了
service/httpd exposed
[root@master ~]# kubectl get pod,svc											#对外暴露端口
NAME                         READY   STATUS    RESTARTS   AGE
pod/httpd-757fb56c8d-w42l5   1/1     Running   0          39s

NAME                 TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
service/httpd        NodePort    10.102.83.215   <none>        80:30176/TCP     26s			#30176端口就是对外映射的端口
service/kubernetes   ClusterIP   10.96.0.1       <none>        443/TCP          112m
[root@master ~]# 
#作为初学者，以上命令先不用纠结，端口就写80即可，如果你写其他的端口可能防火墙拦截了，网页就访问不了

Web page test access can be accessed by using the IP of the master node or the IP of the node node. The port is 30176, as shown below, which means that our k8s has been deployed and the network is ok.