The first part (binary deployment of k8s cluster---preparing the architecture)

1. This article mainly explains the preparation work and installation architecture diagram of kubernetes cluster.

Cluster architecture diagram:

Server list:

2. Server preparation

Upgrade the kernel:

yum -y install kernel-ml-5.7.8-1.el7.elrepo.x86_64.rpm kernel-ml-devel-5.7.8-1.el7.elrepo.x86_64.rpm

调整默认启动内核 
 cat /boot/grub2/grub.cfg | grep  menuentry
 查看是否设置成功
 grub2-editenv list

 重启服务器reboot

3. Turn on ipvs support

cat > /etc/sysconfig/modules/ipvs.modules <<EOF
#!/bin/bash
ipvs_modules="ip_vs ip_vs_lc ip_vs_wlc ip_vs_rr ip_vs_wrr ip_vs_lblc ip_vs_lblcr ip_vs_dh ip_vs_sh ip_vs_fo ip_vs_nq ip_vs_sed ip_vs_ftp"
for kernel_module in \${ipvs_modules}; do
    /sbin/modinfo -F filename \${kernel_module} > /dev/null 2>&1
    if [ $? -eq 0 ]; then
        /sbin/modprobe \${kernel_module}
    fi
done
EOF

chmod 755 /etc/sysconfig/modules/ipvs.modules

bash /etc/sysconfig/modules/ipvs.modules

lsmod | grep ip_vs

Turn off the firewall:

systemctl stop firewalld  && systemctl disable firewalld

Close the swap partition:

swapoff -a  && sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab

Turn off SELinux

setenforce 0
sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config

Modify the number of file handles

vim /etc/security/limits.conf
* soft nofile 65536
* hard nofile 65536
* soft nproc 65536
* hard nproc 65536
* soft  memlock  unlimited
* hard memlock  unlimited

Modify system parameters

vim /etc/sysctl.d/k8s.conf
net.ipv4.tcp_keepalive_time = 600
net.ipv4.tcp_keepalive_intvl = 30
net.ipv4.tcp_keepalive_probes = 10
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1
net.ipv4.neigh.default.gc_stale_time = 120
net.ipv4.conf.all.rp_filter = 0
net.ipv4.conf.default.rp_filter = 0
net.ipv4.conf.default.arp_announce = 2
net.ipv4.conf.lo.arp_announce = 2
net.ipv4.conf.all.arp_announce = 2
net.ipv4.ip_forward = 1
net.ipv4.tcp_max_tw_buckets = 5000
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_max_syn_backlog = 1024
net.ipv4.tcp_synack_retries = 2
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.netfilter.nf_conntrack_max = 2310720
fs.inotify.max_user_watches=89100
fs.may_detach_mounts = 1
fs.file-max = 52706963
fs.nr_open = 52706963
net.bridge.bridge-nf-call-arptables = 1
vm.swappiness = 0
vm.overcommit_memory=1
vm.panic_on_oom=0

sysctl --system

In the production environment, it is recommended to reserve memory to avoid running out of memory and causing ssh to fail to connect to the host:
(2G for 32G machines, 3G for 251, 5G for 500G). The following is reserved 3G

echo 'vm.min_free_kbytes=3000000' >> /etc/sysctl.conf
sysctl -p

Host file preparation: (reserved for more things later)

[root@k8s-master1 nginx-web]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         local-alhost.localdomain localhost6 localhost6.localdomain6
192.168.206.31   k8s-master1
192.168.206.32   k8s-master2
192.168.206.33   k8s-master3
192.168.206.41   k8s-node1
192.168.206.42   k8s-node2
192.168.206.43   k8s-node3
192.168.206.44   k8s-node4
192.168.206.45   k8s-node5
192.168.206.46   k8s-node6
192.168.206.47   k8s-node7
192.168.206.48   k8s-node8
192.168.206.49   k8s-node9

Server time synchronization:

yum -y install ntp ntpdate
ntpdate 0.asia.pool.ntp.org
hwclock --systohc       
systemctl enable ntpd         
systemctl start ntpd
systemctl status ntpd