CentOS-7.X installs k8s cluster through yum

CentOS-7.X installs k8s cluster through yum

VmWare16pro install CentOS-7.9

Download the CentOS-7.9 image iso from the official website

Minimize installation

Virtual machine editing network

Select VMNet8, the NAT mode subnet IP is set to 10.0.0.0, and the subnet mask is set to 255.255.255.0

NAT settings, the gateway address is set to 10.0.0.254

The host VMnet8 address is 10.0.0.1, and the hop is automatically adjusted to 10.

Install CentOS-7.X template machine

Set up template machine network

The IP address of the template machine is set to 10.0.0.10

vi /etc/sysconfig/network-scripts/ifcfg-ens33
BOOTPROTO=static
ONBOOT=yes
IPADDR=10.0.0.10
GATEWAY=10.0.0.254
NETMASK=255.255.255.0
DNS1=10.0.0.254

Close NetworkManager (important)

systemctl disable NetworkManager
systemctl stop NetworkManager
systemctl restart network
#保证能ping通baidu.com

Set hostname

vi /etc/hostname      
CentOS7.9
#或者用下面命令
hostnamectl set-hostname CentOS7.9
#设置完可以hostname查看主机名

Install common commands

yum install -y wget net-tools telnet tree nmap sysstat lrzsz dos2unix bind-utils vim less
yum -y install bash-completion
yum -y install epel-release
yum -y install yum-plugin-fastestmirror
source /etc/profile.d/bash_completion.sh

Switch the source of Alibaba Cloud

mkdir -p /etc/yum.repos.d/back/
mv /etc/yum.repos.d/*.repo /etc/yum.repos.d/back/
wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo
wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo

Turn off the firewall

systemctl stop firewalld
systemctl disable firewalld
yum -y install iptables-services
systemctl start iptables 
systemctl enable iptables 
iptables -F
service iptables save

Close selinux

swapoff -a
sed -ri 's/.*swap.*/#&/' /etc/fstab
#其实就是找到swap那一行，在开头加#

Adjust time zone

# 设置系统时区为中国/上海 
timedatectl set-timezone Asia/Shanghai
# 将当前的UTC时间写入硬件时钟 
timedatectl set-local-rtc 0
# 重启依赖于系统时间的服务 
systemctl restart rsyslog 
systemctl restart crond

Turn off services that the system does not need

systemctl stop postfix 
systemctl disable postfix

Set the system default log systemd journald (originally rsyslogd)

mkdir /var/log/journal #持久化保存日志的目录
mkdir /etc/systemd/journald.conf.d
cat > /etc/systemd/journald.conf.d/99-prophet.conf <<EOF
[Journal]
# 持久化保存到磁盘
Storage=persistent
# 压缩历史日志
Compress=yes
SyncIntervalSec=5m
RateLimitInterval=30s
RateLimitBurst=1000
# 最大占用空间10G
SystemMaxUse=10G
# 单日志文件最大200M
SystemMaxFileSize=200M
# 日志保存时间2周
MaxRetentionSec=2week
# 不将日志转发到syslog
ForwardToSyslog=no
EOF
systemctl restart systemd-journald

Upgrade system kernel

 rpm -Uvh http://www.elrepo.org/elrepo-release-7.0-3.el7.elrepo.noarch.rpm
 yum --disablerepo="*" --enablerepo="elrepo-kernel" list available
 yum --enablerepo=elrepo-kernel install kernel-ml
 awk -F\' '$1=="menuentry " {print $2}' /etc/grub2.cfg
 cat /etc/grub2.cfg
 grub2-set-default 0
 awk -F\' '$1=="menuentry " {print $2}' /etc/grub2.cfg
#默认升到了5.11
reboot

Configure k8s parameters (it does not seem to be needed for installation via yum)

vi /etc/sysctl.d/kubernetes.conf
net.bridge.bridge-nf-call-iptables=1
net.bridge.bridge-nf-call-ip6tables=1
net.ipv4.ip_forward=1
#net.ipv4.tcp_tw_recycle=0
vm.swappiness=0 #禁止使用swap空间，只有当系统OOM时才允许使用它
vm.overcommit_memory=1 #不检查物理内存是否够用
vm.panic_on_oom=0 # 开启 OOM
fs.inotify.max_user_instances=8192
fs.inotify.max_user_watches=1048576
fs.file-max=52706963
fs.nr_open=52706963
net.ipv6.conf.all.disable_ipv6=1
net.netfilter.nf_conntrack_max=2310720

#刷新配置
sysctl -p /etc/sysctl.d/kubernetes.conf
#出现报错  sysctl: cannot stat /proc/sys/net/netfilter/nf_conntrack_max: No such file or directory
lsmod |grep conntrack
modprobe ip_conntrack
lsmod |grep conntrack
sysctl -p /etc/sysctl.d/kubernetes.conf

Create k8s cluster

The k8s cluster is divided into master node and node node. In this case, the master node acts as both a master and a node. The services they install are as follows:

master

• etcd
• api-server
• controller-manager
• scheduler
• kubelet
• kube-proxy
• docker (install kubelet will automatically install)

node

• kubelet
• kube-proxy
• docker (install kubelet will automatically install)

Clone copy

Use the newly configured machine as a template machine to create 3 clone copies. In order to save space, choose to create a linked clone here.

The 3 machines after cloning are as follows:

• master-10.0.0.11
• node01-10.0.0.21
• node02-10.0.0.22

Modify the ip addresses of 3 machines separately

vi /etc/sysconfig/network-scripts/ifcfg-ens33
IPADDR=10.0.0.11
IPADDR=10.0.0.21
IPADDR=10.0.0.22

Modify the host names of 3 machines separately

hostnamectl set-hostname master
hostnamectl set-hostname node01
hostnamectl set-hostname node02

Modify the hosts file of the master machine

vi /etc/hosts
10.0.0.11 master
10.0.0.21 node01
10.0.0.22 node02

Transfer the hosts file of the master machine to the node node

scp -rp /etc/hosts 10.0.0.21:/etc/hosts
scp -rp /etc/hosts 10.0.0.22:/etc/hosts
#第一次使用scp命令，需要输入密码

Build master node and 2 node nodes

Install etcd service

#只需要在master节点装
yum install etcd -y
#修改etcd配置
vi /etc/etcd/etcd.conf
#搜索=左侧的key，修改成以下内容
#etcd没有配置集群，只修改以下2行即可
ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:2379"
ETCD_ADVERTISE_CLIENT_URLS="http://10.0.0.11:2379"
#启动etcd且设置开机自启
systemctl start etcd
systemctl enable etcd
netstat -lntup

Install kubernetes-master

yum -y install kubernetes-master
#配置apiserver
vi /etc/kubernetes/apiserver
#搜索=左侧的key，修改成以下内容
KUBE_API_ADDRESS="--insecure-bind-address=0.0.0.0"
KUBE_API_PORT="--port=8080"
KUBELET_PORT="--kubelet-port=10250"
KUBE_ETCD_SERVERS="--etcd-servers=http://10.0.0.11:2379"

#配置controller-manager和scheduler kubelet和kube-proxy共用该配置文件
vi /etc/kubernetes/config
KUBE_MASTER="--master=http://10.0.0.11:8080"

Start service

systemctl start kube-apiserver.service
systemctl start kube-controller-manager.service
systemctl start kube-scheduler.service
systemctl enable kube-apiserver.service
systemctl enable kube-controller-manager.service
systemctl enable kube-scheduler.service
kubectl get componentstatus

Install kubernetes-node on the master node and 2 node nodes

yum install kubernetes-node -y
#kubernetes-node依赖docker，所以会自动安装docker
#master节点修改kubelet配置
vi /etc/kubernetes/kubelet
KUBELET_ADDRESS="--address=10.0.0.11"
KUBELET_PORT="--port=10250"
KUBELET_HOSTNAME="--hostname-override=master"
KUBELET_API_SERVER="--api-servers=http://10.0.0.11:8080"
#启动master节点的kubelet
systemctl start kubelet.service
systemctl enable kubelet.service
systemctl start kube-proxy.service
systemctl enable kube-proxy.service

kubectl get nodes
#配置两个node节点
vi /etc/kubernetes/config
KUBE_MASTER="--master=http://10.0.0.11:8080"
#配置node01节点
vi /etc/kubernetes/kubelet
KUBELET_ADDRESS="--address=10.0.0.21"
KUBELET_PORT="--port=10250"
KUBELET_HOSTNAME="--hostname-override=node01"
KUBELET_API_SERVER="--api-servers=http://10.0.0.11:8080"
#配置node02节点
vi /etc/kubernetes/kubelet
KUBELET_ADDRESS="--address=10.0.0.22"
KUBELET_PORT="--port=10250"
KUBELET_HOSTNAME="--hostname-override=node02"
KUBELET_API_SERVER="--api-servers=http://10.0.0.11:8080"
#启动2个node节点的kubelet
systemctl start kubelet.service
systemctl enable kubelet.service
systemctl start kube-proxy.service
systemctl enable kube-proxy.service

Configure flannel network

#所有节点安装flannel
yum install flannel -y
vi /etc/sysconfig/flanneld
FLANNEL_ETCD_ENDPOINTS="http://10.0.0.11:2379"
FLANNEL_ETCD_PREFIX="/atomic.io/network"
#master设置etcd配置
etcdctl set /atomic.io/network/config '{"Network":"172.16.0.0/16"}'
#所有节点启动flannel
systemctl start flanneld.service
systemctl enable flanneld.service
systemctl restart docker

Set up docker Alibaba Cloud image acceleration

#先保证docker本身没有问题，pull镜像以及运行容器都是ok的，后面k8s运行容器有问题，会比较好排查
sudo mkdir -p /etc/docker
sudo tee /etc/docker/daemon.json <<-'EOF'
{
  "registry-mirrors": ["https://zhjxhme4.mirror.aliyuncs.com"]
}
EOF
sudo systemctl daemon-reload
sudo systemctl restart docker

test

docker pull busybox
docker run -it busybox
ip addr
#互相ping端口，由于一开始就把iptables规则都清空了，所以直接就能ping通