CentOS-7.X通过yum安装k8s集群

VmWare16pro安装CentOS-7.9

官网下载CentOS-7.9镜像iso

选择最小化安装

虚拟机编辑网络

选择VMNet8，NAT模式子网IP设为 10.0.0.0，子网掩码设为255.255.255.0

NAT设置，网关地址设置为10.0.0.254

宿主机VMnet8地址 10.0.0.1,自动跃点调到10

安装CentOS-7.X模板机

设置模板机网络

模板机IP地址设置为10.0.0.10

vi /etc/sysconfig/network-scripts/ifcfg-ens33
BOOTPROTO=static
ONBOOT=yes
IPADDR=10.0.0.10
GATEWAY=10.0.0.254
NETMASK=255.255.255.0
DNS1=10.0.0.254

关闭NetworkManager （重要）

systemctl disable NetworkManager
systemctl stop NetworkManager
systemctl restart network
#保证能ping通baidu.com

设置主机名

vi /etc/hostname      
CentOS7.9
#或者用下面命令
hostnamectl set-hostname CentOS7.9
#设置完可以hostname查看主机名

安装常用命令

yum install -y wget net-tools telnet tree nmap sysstat lrzsz dos2unix bind-utils vim less
yum -y install bash-completion
yum -y install epel-release
yum -y install yum-plugin-fastestmirror
source /etc/profile.d/bash_completion.sh

切换阿里云的源

mkdir -p /etc/yum.repos.d/back/
mv /etc/yum.repos.d/*.repo /etc/yum.repos.d/back/
wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo
wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo

关闭防火墙

systemctl stop firewalld
systemctl disable firewalld
yum -y install iptables-services
systemctl start iptables 
systemctl enable iptables 
iptables -F
service iptables save

关闭selinux

swapoff -a
sed -ri 's/.*swap.*/#&/' /etc/fstab
#其实就是找到swap那一行，在开头加#

调整时区

# 设置系统时区为中国/上海 
timedatectl set-timezone Asia/Shanghai
# 将当前的UTC时间写入硬件时钟 
timedatectl set-local-rtc 0
# 重启依赖于系统时间的服务 
systemctl restart rsyslog 
systemctl restart crond

关闭系统不需要的服务

systemctl stop postfix 
systemctl disable postfix

设置系统默认日志systemd journald（原本为rsyslogd ）

mkdir /var/log/journal #持久化保存日志的目录
mkdir /etc/systemd/journald.conf.d
cat > /etc/systemd/journald.conf.d/99-prophet.conf <<EOF
[Journal]
# 持久化保存到磁盘
Storage=persistent
# 压缩历史日志
Compress=yes
SyncIntervalSec=5m
RateLimitInterval=30s
RateLimitBurst=1000
# 最大占用空间10G
SystemMaxUse=10G
# 单日志文件最大200M
SystemMaxFileSize=200M
# 日志保存时间2周
MaxRetentionSec=2week
# 不将日志转发到syslog
ForwardToSyslog=no
EOF
systemctl restart systemd-journald

升级系统内核

 rpm -Uvh http://www.elrepo.org/elrepo-release-7.0-3.el7.elrepo.noarch.rpm
 yum --disablerepo="*" --enablerepo="elrepo-kernel" list available
 yum --enablerepo=elrepo-kernel install kernel-ml
 awk -F\' '$1=="menuentry " {print $2}' /etc/grub2.cfg
 cat /etc/grub2.cfg
 grub2-set-default 0
 awk -F\' '$1=="menuentry " {print $2}' /etc/grub2.cfg
#默认升到了5.11
reboot

配置k8s参数（通过yum方式安装好像不需要）

vi /etc/sysctl.d/kubernetes.conf
net.bridge.bridge-nf-call-iptables=1
net.bridge.bridge-nf-call-ip6tables=1
net.ipv4.ip_forward=1
#net.ipv4.tcp_tw_recycle=0
vm.swappiness=0 #禁止使用swap空间，只有当系统OOM时才允许使用它
vm.overcommit_memory=1 #不检查物理内存是否够用
vm.panic_on_oom=0 # 开启 OOM
fs.inotify.max_user_instances=8192
fs.inotify.max_user_watches=1048576
fs.file-max=52706963
fs.nr_open=52706963
net.ipv6.conf.all.disable_ipv6=1
net.netfilter.nf_conntrack_max=2310720

#刷新配置
sysctl -p /etc/sysctl.d/kubernetes.conf
#出现报错  sysctl: cannot stat /proc/sys/net/netfilter/nf_conntrack_max: No such file or directory
lsmod |grep conntrack
modprobe ip_conntrack
lsmod |grep conntrack
sysctl -p /etc/sysctl.d/kubernetes.conf

创建k8s集群

k8s集群中分master节点和node节点，本案例中master节点既充当master又充当node，它们安装的服务如下：

master

etcd
api-server
controller-manager
scheduler
kubelet
kube-proxy
docker（安装kubelet会自动安装）

node

kubelet
kube-proxy
docker（安装kubelet会自动安装）

克隆副本

以刚配置好的机器当做模板机，创建3个克隆副本，为节省空间，这里选择创建链接克隆。

克隆完成后的3台机器如下：

master-10.0.0.11
node01-10.0.0.21
node02-10.0.0.22

分别修改3台机器的ip地址

vi /etc/sysconfig/network-scripts/ifcfg-ens33
IPADDR=10.0.0.11
IPADDR=10.0.0.21
IPADDR=10.0.0.22

分别修改3台机器主机名

hostnamectl set-hostname master
hostnamectl set-hostname node01
hostnamectl set-hostname node02

修改master机器的hosts文件

vi /etc/hosts
10.0.0.11 master
10.0.0.21 node01
10.0.0.22 node02

传输master机器的hosts文件到node节点

scp -rp /etc/hosts 10.0.0.21:/etc/hosts
scp -rp /etc/hosts 10.0.0.22:/etc/hosts
#第一次使用scp命令，需要输入密码

搭建master节点和2个node节点

安装etcd服务

#只需要在master节点装
yum install etcd -y
#修改etcd配置
vi /etc/etcd/etcd.conf
#搜索=左侧的key，修改成以下内容
#etcd没有配置集群，只修改以下2行即可
ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:2379"
ETCD_ADVERTISE_CLIENT_URLS="http://10.0.0.11:2379"
#启动etcd且设置开机自启
systemctl start etcd
systemctl enable etcd
netstat -lntup

安装kubernetes-master

yum -y install kubernetes-master
#配置apiserver
vi /etc/kubernetes/apiserver
#搜索=左侧的key，修改成以下内容
KUBE_API_ADDRESS="--insecure-bind-address=0.0.0.0"
KUBE_API_PORT="--port=8080"
KUBELET_PORT="--kubelet-port=10250"
KUBE_ETCD_SERVERS="--etcd-servers=http://10.0.0.11:2379"

#配置controller-manager和scheduler kubelet和kube-proxy共用该配置文件
vi /etc/kubernetes/config
KUBE_MASTER="--master=http://10.0.0.11:8080"

启动服务

systemctl start kube-apiserver.service
systemctl start kube-controller-manager.service
systemctl start kube-scheduler.service
systemctl enable kube-apiserver.service
systemctl enable kube-controller-manager.service
systemctl enable kube-scheduler.service
kubectl get componentstatus

master节点和2个node节点安装kubernetes-node

yum install kubernetes-node -y
#kubernetes-node依赖docker，所以会自动安装docker
#master节点修改kubelet配置
vi /etc/kubernetes/kubelet
KUBELET_ADDRESS="--address=10.0.0.11"
KUBELET_PORT="--port=10250"
KUBELET_HOSTNAME="--hostname-override=master"
KUBELET_API_SERVER="--api-servers=http://10.0.0.11:8080"
#启动master节点的kubelet
systemctl start kubelet.service
systemctl enable kubelet.service
systemctl start kube-proxy.service
systemctl enable kube-proxy.service

kubectl get nodes
#配置两个node节点
vi /etc/kubernetes/config
KUBE_MASTER="--master=http://10.0.0.11:8080"
#配置node01节点
vi /etc/kubernetes/kubelet
KUBELET_ADDRESS="--address=10.0.0.21"
KUBELET_PORT="--port=10250"
KUBELET_HOSTNAME="--hostname-override=node01"
KUBELET_API_SERVER="--api-servers=http://10.0.0.11:8080"
#配置node02节点
vi /etc/kubernetes/kubelet
KUBELET_ADDRESS="--address=10.0.0.22"
KUBELET_PORT="--port=10250"
KUBELET_HOSTNAME="--hostname-override=node02"
KUBELET_API_SERVER="--api-servers=http://10.0.0.11:8080"
#启动2个node节点的kubelet
systemctl start kubelet.service
systemctl enable kubelet.service
systemctl start kube-proxy.service
systemctl enable kube-proxy.service

配置flannel网络

#所有节点安装flannel
yum install flannel -y
vi /etc/sysconfig/flanneld
FLANNEL_ETCD_ENDPOINTS="http://10.0.0.11:2379"
FLANNEL_ETCD_PREFIX="/atomic.io/network"
#master设置etcd配置
etcdctl set /atomic.io/network/config '{"Network":"172.16.0.0/16"}'
#所有节点启动flannel
systemctl start flanneld.service
systemctl enable flanneld.service
systemctl restart docker

设置docker阿里云镜像加速

#先保证docker本身没有问题，pull镜像以及运行容器都是ok的，后面k8s运行容器有问题，会比较好排查
sudo mkdir -p /etc/docker
sudo tee /etc/docker/daemon.json <<-'EOF'
{
  "registry-mirrors": ["https://zhjxhme4.mirror.aliyuncs.com"]
}
EOF
sudo systemctl daemon-reload
sudo systemctl restart docker

测试

docker pull busybox
docker run -it busybox
ip addr
#互相ping端口，由于一开始就把iptables规则都清空了，所以直接就能ping通