Microservices: a module is divided into multiple modules
Distributed: multiple machines form one machine
Day01 K8S installation and deployment
1. Host related configuration
1 Close selinux, close防火墙
1.1 Reasons for firewalls ( nftablesback-end compatibility issues, duplicate firewall rules)
1.2 About selinuxthe reason (closed selinuxto allow the container to access the host's file system)
# 永久关闭
sed -i 's#enforcing#disabled#g' /etc/sysconfig/selinux
# 零时关闭
setenforce 0
`vi /etc/selinux/config`
1.3 swapThis linuxwill be used automatically when the memory is insufficient, and swapsome memory data will be stored in the disk. This will reduce the performance, and it is recommended to turn it off for performance considerations
swapoff -a
sed -i.bak 's/^.*centos-swap/#&/g' /etc/fstab
echo 'KUBELET_EXTRA_ARGS="--fail-swap-on=false"' > /etc/sysconfig/kubelet
2 Configure the basic yum source
2.1 Backup image
mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.backup
2.2 Download and install
curl -o /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-7.repo
sed -i -e '/mirrors.cloud.aliyuncs.com/d' -e '/mirrors.aliyuncs.com/d' /etc/yum.repos.d/CentOS-Base.repo
3 refresh cache
yum makecache # 实质是把镜像源的文件名拉下来，加速本地安装包的安装
4 Update the kernel
yum -y install wget # 安装 wget工具
yum update --exclud=kernel* -y # 更新内核
5 Upgrade the kernel version
Since Docker operation requires newer system kernel functions, such as ipvs, etc., in general, we need to use the system kernel version 4.0+ or higher.
The kernel requirement is 4.18+, if it is, there is
CentOS 8no need to upgrade the kernel
wget https://elrepo.org/linux/kernel/el7/x86_64/RPMS/kernel-lt-devel-4.4.245-1.el7.elrepo.x86_64.rpm
wget https://elrepo.org/linux/kernel/el7/x86_64/RPMS/kernel-lt-4.4.245-1.el7.elrepo.x86_64.rpm
yum localinstall -y kernel-lt* # 本地安装
grub2-set-default 0 && grub2-mkconfig -o /etc/grub2.cfg #设置为默认启动第一个内核,重新生成新的内核配置文件
grubby --default-kernel
reboot # 重启
Two, install dependent components
2.1 IPVS installation
# 下载IPVS模块
yum install -y conntrack-tools ipvsadm ipset conntrack libseccomp
# 加载IPVS模块
cat > /etc/sysconfig/modules/ipvs.modules <<EOF
#!/bin/bash
ipvs_modules="ip_vs ip_vs_lc ip_vs_wlc ip_vs_rr ip_vs_wrr ip_vs_lblc ip_vs_lblcr
ip_vs_dh ip_vs_sh ip_vs_fo ip_vs_nq ip_vs_sed ip_vs_ftp nf_conntrack"
for kernel_module in \${ipvs_modules}; do
/sbin/modinfo -F filename \${kernel_module} > /dev/null 2>&1
if [ $? -eq 0 ]; then
/sbin/modprobe \${kernel_module}
fi
done
EOF
#
chmod 755 /etc/sysconfig/modules/ipvs.modules && bash /etc/sysconfig/modules/ipvs.modules && lsmod | grep ip_vs
2.2 Kernel parameter tuning
cat > /etc/sysctl.d/k8s.conf << EOF
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
fs.may_detach_mounts = 1
vm.overcommit_memory=1
vm.panic_on_oom=0
fs.inotify.max_user_watches=89100
fs.file-max=52706963
fs.nr_open=52706963
net.ipv4.tcp_keepalive_time = 600
net.ipv4.tcp.keepaliv.probes = 3
net.ipv4.tcp_keepalive_intvl = 15
net.ipv4.tcp.max_tw_buckets = 36000
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp.max_orphans = 327680
net.ipv4.tcp_orphan_retries = 3
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_max_syn_backlog = 16384
net.ipv4.ip_conntrack_max = 65536
net.ipv4.tcp_max_syn_backlog = 16384
net.ipv4.top_timestamps = 0
net.core.somaxconn = 16384
EOF
# 立即生效
sysctl --system
Three, install the basic software
yum install wget expect vim net-tools ntp bash-completion ipvsadm ipset jq iptables -y >> install.log 2>&1
Fourth, install dokcer
K8S components depend on docker
yum install -y yum-utils device-mapper-persistent-data lvm2 >> install.log 2>&1
yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo >> install.log 2>&1
yum install docker-ce -y >> install.log 2>&1 # 安装最新版 并写入安装日志（错误，正确）
sudo mkdir -p /etc/docker # 创建文件夹 读取规则
# 设置镜像加速源
tee /etc/docker/daemon.json <<-'EOF'
{
"registry-mirrors": ["https://8mh75mhz.mirror.aliyuncs.com"]
}
EOF
systemctl daemon-reload ; systemctl restart docker;systemctl enable --now docker.service >> install.log 2>&1
# 分号相当于 && 重新加载镜像配置 重启docker 设置docker默认启动 写入日志
echo '设置docker开机启动' # 给个提醒
echo '设置docker开机启动'
Five, synchronize the cluster time
yum install ntp -y # 安装 时间插件
ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime # 软连接 使用上海时间
echo 'Asia/Shanghai' > /etc/timezone
ntpdate time2.aliyun.com # 同步时间
Six, configure Kubernetes source
cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
setenforce 0 # 检查是否关闭selinux
6.2 Installation kubelet kubeadm kubectl
yum install -y kubelet kubeadm kubectl
6.3 Start and set automatic start
systemctl enable kubelet && systemctl start kubelet
-----------------------------------------------------------------------------------------------------------
Created symlink from /etc/systemd/system/multi-user.target.wants/kubelet.service to /usr/lib/systemd/system/kubelet.service.
# 没有提示也可以
At this point, the node machine task is basically completed. If notready appears because there is no network plug-in, you can skip to step ten, and then the node machine will join the master immediately through the token
Seven, node initialization (master host configuration is sufficient)
kubeadm init \
--image-repository=registry.cn-hangzhou.aliyuncs.com/k8s2me \
--service-cidr=10.96.0.0/12 \
--pod-network-cidr=10.244.0.0/16
Eight, configure kubernetes user information
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
Nine, edit node
vim /etc/hosts
172.16.0.50 kubernetes-master-01 # 端口名和节点名
172.16.0.53 kubernetes-node-01
172.16.0.54 kubernetes-node-02
get nodes can only be executed on the master node, the child nodes do not need to be executed
notreday是由于没有网络插件的原因
Note: If it is installed, it cannot be used after restarting. Generally, docker has not started, and you need to re-customize the node, and then set both kub and docker to self-start
systemctl enable --now docker.service # docker自启
10. Install the cluster network plug-in
docker pull registry.cn-hangzhou.aliyuncs.com/k8sos/flannel:v0.12.0-amd64 ;\
docker tag registry.cn-hangzhou.aliyuncs.com/k8sos/flannel:v0.12.0-amd64 \
quay.io/coreos/flannel:v0.12.0-amd64
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
kubectl get pods -n kube-system # 命令来查询每个 Pod 的更多信息
------------------------------------------------------------------------
NAME READY STATUS RESTARTS AGE
coredns-7dcc599b9f-gn7cs 1/1 Running 1 15h
coredns-7dcc599b9f-s94x7 1/1 Running 1 15h
etcd-k8s-master 1/1 Running 1 15h
kube-apiserver-k8s-master 1/1 Running 1 15h
kube-controller-manager-k8s-master 1/1 Running 1 15h
kube-flannel-ds-drgbk 1/1 Running 1 15h # init 代表正在初始化 等待即可
kube-flannel-ds-gcwbr 1/1 Running 0 13h
kube-flannel-ds-tchxk 1/1 Running 0 13h
kube-proxy-brx4c 1/1 Running 1 15h
kube-proxy-mwk4w 1/1 Running 0 13h
kube-proxy-qspfb 1/1 Running 0 13h
kube-scheduler-k8s-master 1/1 Running 1 15h
参数 -w whatch 监听
会一只停留 ctrl+c退出
11. Node node joins the cluster
# master机器创建
# 创建 TOKEN
kubeadm token create --print-join-command
------------------------------------------------------------------------------------------------------------
W1207 20:57:38.878150 70266 configset.go:348] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io]
kubeadm join 10.0.0.80:6443 --token 7fsv9l.zydosm4486o8aatk --discovery-token-ca-cert-hash sha256:cea884131c7cb53b477f5bc42090072ba1e4a2ec81c34d29c27d743dd07497b0
# 节点机器 token的有效期是24小时
kubeadm join 10.0.0.80:6443 --token 7fsv9l.zydosm4486o8aatk --discovery-token-ca-cert-hash sha256:cea884131c7cb53b477f5bc42090072ba1e4a2ec81c34d29c27d743dd07497b0 # 加入配置
Do not operate any nodes at this time
12. End cluster uninstallation cleanup
If the port is occupied, it can be re-cleaned, installed and initialized.
kubeadm reset -f
modprobe -r ipip
lsmod
rm -rf ~/.kube/
rm -rf /etc/kubernetes/
rm -rf /etc/systemd/system/kubelet.service.d
rm -rf /etc/systemd/system/kubelet.service
rm -rf /usr/bin/kube*
rm -rf /etc/cni
rm -rf /opt/cni
rm -rf /var/lib/etcd
rm -rf /var/etcd
yum clean all
yum remove kube*
Note: If the reinstallation kubelet kubeadm kubectlalready exists, you need to manually delete these three plugins
yum remove kubelet kubeadm kubectl
Then just reinstall
13. Test whether the cluster DNS is normal
kubectl run test -it --rm --image=busybox:1.28.3
nslookup kubernetes
If the following figure appears, the cluster is normal
error:
Troubleshooting
1. Check the environment variables
Env | grep -i cube
2. Check the docker service
systemctl status docker.service
3. Check kubelet service
systemctl status kubelet.service
1. Port 6443 problem after restarting (sawpoff problem)
[root@k8s-master ~]# kubectl get node
The connection to the server 10.0.0.80:6443 was refused - did you specify the right host or port?
# 通过 systemctl status kubelet.service 查看状态
[root@k8s-master ~]systemctl status kubelet.service
● kubelet.service - kubelet: The Kubernetes Node Agent
Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled; vendor preset: disabled)
Drop-In: /usr/lib/systemd/system/kubelet.service.d
└─10-kubeadm.conf
Active: activating (auto-restart) (Result: exit-code) since Tue 2020-12-08 22:19:05 CST; 6s ago
Docs: https://kubernetes.io/docs/
Process: 1949 ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS (code=exited, status=255)
Main PID: 1949 (code=exited, status=255) # 这里出现错误
#关闭swapoff分区关闭 重启即可
swapoff -a
systemctl restart kubelet.service