文章目录
K8S 集群的安装
参考文章
操作系统准备
查看和升级系统内核
[root@k8s-master k8s]# uname -r
3.10.0-1160.el7.x86_64
rpm -import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org
rpm -Uvh http://www.elrepo.org/elrepo-release-7.0-3.el7.elrepo.noarch.rpm
yum -y --enablerepo=elrepo-kernel install kernel-ml.x86_64 kernel-ml-devel.x86_64
安装图形界面
因为K8s 的网络实在是太复杂了，我先随便找一个节点，安装一下图形界面，便于后面在该主机上测试访问服务的问题。
启动图形界面后，操作还是卡，所以不要修改默认进入图形界面，只在需要的时候，通过 startx 命令进去看一下就好了
yum groupinstall “GNOME Desktop” “Graphical Administration Tools”
startx
配置系统环境变量
cat <<EOF >/etc/sysctl.d/k8s.conf
vm.swappiness=0
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
EOF
cat > /etc/sysconfig/modules/ipvs.modules <<EOF
#!/bin/bash
modprobe -- ip_vs
modprobe -- ip_vs_rr
modprobe -- ip_vs_wrr
modprobe -- ip_vs_sh
modprobe -- nf_conntrack
EOF
安装Docker 和 K8S
yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
yum update && yum install -y docker-ce
cat > /etc/docker/daemon.json <<EOF
{
"exec-opts": ["native.cgroupdriver=systemd"],
"log-driver": "json-file",
"log-opts": {
"max-size": "100m"
}
}
EOF
>>
cat > /etc/yum.repos.d/kubernetes.repo <<EOF
[kubernetes]
name=kubernetes
baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
gpgkey=http://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
EOF
yum install -y kubeadm kubectl kubelet
// 20210911 最新版本 1.22.1
集群初始化
kubeadm init --kubernetes-version=v1.22.1 --pod-network-cidr=10.244.0.0/16
开始的时候Copy网上提供的参数，错误的Copy了参数--apiserver-advertise-address=10.211.55.46,导致后续出现一堆错误的情况。记录一下TroubleShooting 方法
- 检查kubelet是否正常启动，
systemctl status kubelet, 然后查看服务日志journalctl -xeu kubelet docker ps -a检查POD状态是否正常，手动重启Docker 是systemctl start/stop docker.socket， 清理container命令:docker container prune- 上一次运行失败的时候，记得要先清理上一次配置失败遗留下来的错误信息
kubeadm reset或者进行手动清理
rm -rf /etc/kubernetes/*
rm -rf /var/lib/etcd
systemctl stop kubelet
NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized因为kubelet配置了network-plugin=cni，但是还没安装，所以状态会是NotReady,不想看这个报错或者不需要网络，就可以修改kubelet配置文件，去掉network-plugin=cni 就可以了。也可以提前安装 network-plugin
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
# 也可以尝试设置环境变量试一下 export KUBECONFIG=/etc/kubernetes/admin.conf
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
添加集群节点
kubeadm join 172.16.208.132:6443 --token bhybvb.2t30d7kmn9oqkgl4
–discovery-token-ca-cert-hash sha256:434ef9085792043cf4745eb8becff091c1415c15b087bc361b50f7c498f13761
检查安装结果
[root@k8s-master kubelet]# kubectl get cs
Warning: v1 ComponentStatus is deprecated in v1.19+
NAME STATUS MESSAGE ERROR
scheduler Unhealthy Get "http://127.0.0.1:10251/healthz": dial tcp 127.0.0.1:10251: connect: connection refused
controller-manager Healthy ok
etcd-0 Healthy {"health":"true","reason":""}
参考解决: https://blog.csdn.net/cymm_liu/article/details/108458197 解决kubernetes:v1.18.6-1.19.0 get cs127.0.0.1 connection refused错误
[root@k8s-master manifests]# kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
k8s-master Ready control-plane,master 63m v1.22.1 172.16.208.132 <none> CentOS Linux 7 (Core) 5.14.2-1.el7.elrepo.x86_64 docker://20.10.8
k8s-node1 Ready <none> 4m53s v1.22.1 172.16.208.133 <none> CentOS Linux 7 (Core) 5.14.2-1.el7.elrepo.x86_64 docker://20.10.8
k8s-node2 Ready <none> 3m35s v1.22.1 172.16.208.134 <none> CentOS Linux 7 (Core) 5.14.2-1.el7.elrepo.x86_64 docker://20.10.8
k8s-node3 Ready <none> 3m32s v1.22.1 172.16.208.135 <none> CentOS Linux 7 (Core) 5.14.2-1.el7.elrepo.x86_64 docker://20.10.8
[root@k8s-master manifests]# kubectl get pods -o wide -n kube-system
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
coredns-78fcd69978-tfpbd 0/1 Running 8 (80s ago) 63m 10.244.0.2 k8s-master <none> <none>
coredns-78fcd69978-vpftt 0/1 Running 9 (2s ago) 63m 10.244.0.3 k8s-master <none> <none>
etcd-k8s-master 1/1 Running 3 63m 172.16.208.132 k8s-master <none> <none>
kube-apiserver-k8s-master 1/1 Running 3 63m 172.16.208.132 k8s-master <none> <none>
kube-controller-manager-k8s-master 1/1 Running 3 63m 172.16.208.132 k8s-master <none> <none>
kube-flannel-ds-4dw98 1/1 Running 0 3m44s 172.16.208.134 k8s-node2 <none> <none>
kube-flannel-ds-czphj 1/1 Running 0 5m2s 172.16.208.133 k8s-node1 <none> <none>
kube-flannel-ds-qngvs 1/1 Running 0 3m41s 172.16.208.135 k8s-node3 <none> <none>
kube-flannel-ds-x6f76 1/1 Running 0 17m 172.16.208.132 k8s-master <none> <none>
kube-proxy-f2gmk 1/1 Running 0 63m 172.16.208.132 k8s-master <none> <none>
kube-proxy-jzrvr 1/1 Running 0 3m41s 172.16.208.135 k8s-node3 <none> <none>
kube-proxy-nt869 1/1 Running 0 5m2s 172.16.208.133 k8s-node1 <none> <none>
kube-proxy-s2btj 1/1 Running 0 3m44s 172.16.208.134 k8s-node2 <none> <none>
kube-scheduler-k8s-master 1/1 Running 0 8m15s 172.16.208.132 k8s-master <none> <none>
Kubernets Dashboard UI
此步骤非必选，主要是用于自己熟悉K8S的相关操作而做的工作，没兴趣的可以跳过。
- 官方部署dashboard的服务没使用nodeport，将yaml文件下载到本地,
wget https://raw.githubusercontent.com/kubernetes/dashboard/v2.2.0/aio/deploy/recommended.yaml - 修改 recommended.yaml 中Service 类型为 NodePort类型，且端口为 30000, 这样就可以通过30000 访问dashboard 页面了
kubectl create -f recommended.yaml- 在访问
https://虚拟机:30000地址时，浏览器报错:Your connection is not private，有可能是证书的问题， 选择通过在一台虚拟机上启动安装和启动图形化页面，再访问服务的 30000 - 通过Token 进入管理页面
- 在管理页面中发现查看不了任何东西，查看POD日志
kubectl logs -f -n kubernetes-dashboard kubernetes-dashboard-78c79f97b4-tkck6， 发现是用户没有对应的权限，给对应的用户绑定权限
# 查看Token
[root@k8s-master k8s]# kubectl get secrets -n kubernetes-dashboard
NAME TYPE DATA AGE
default-token-f2bp7 kubernetes.io/service-account-token 3 22h
kubernetes-dashboard-certs Opaque 0 22h
kubernetes-dashboard-csrf Opaque 1 22h
kubernetes-dashboard-key-holder Opaque 2 22h
kubernetes-dashboard-token-77z6g kubernetes.io/service-account-token 3 22h
[root@k8s-master k8s]# kubectl describe secrets -n kubernetes-dashboard kubernetes-dashboard-token-77z6g | grep token | awk 'NR==3{print $2}'
[root@k8s-master k8s]# kubectl get svc -n kubernetes-dashboard
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
dashboard-metrics-scraper ClusterIP 10.98.63.120 <none> 8000/TCP 22h
kubernetes-dashboard NodePort 10.100.246.213 <none> 443:30000/TCP 128m
[root@k8s-master k8s]#
[root@k8s-master k8s]#
[root@k8s-master k8s]# kubectl get pod -n kubernetes-dashboard
NAME READY STATUS RESTARTS AGE
dashboard-metrics-scraper-856586f554-ksvfj 1/1 Running 0 22h
kubernetes-dashboard-78c79f97b4-tkck6 1/1 Running 0 73m
kubectl create clusterrolebinding serviceaccount-cluster-admin2 --clusterrole=cluster-admin --user=system:serviceaccount:kubernetes-dashboard:kubernetes-dashboard
Expose Service
当我们启动了一个POD后，因为网络问题，我们在外部并不能访问到这个服务，此时通过expose 创建一个Service， Service类型是NodePort，此时会将一个Port映射出来，供外部访问
kubectl expose deployment/kubernetes-bootcamp --type="NodePort" --port 8080
kubectl delete service -l app=kubernetes-bootcamp
Objects in kubernetes
NameSpace --> Deployment --> ReplicaSet --> Pod
# 创建deployment
kubectl create deployment --image nginx demo-nginx
[root@k8s-master namespaces]# kubectl get deployment
NAME READY UP-TO-DATE AVAILABLE AGE
demo-nginx 1/1 1 1 7m52s
[root@k8s-master namespaces]# kubectl describe deployment demo-nginx
Name: demo-nginx
Namespace: default
CreationTimestamp: Sun, 12 Sep 2021 16:35:34 +0800
Labels: app=demo-nginx
Annotations: deployment.kubernetes.io/revision: 1
Selector: app=demo-nginx
Replicas: 1 desired | 1 updated | 1 total | 1 available | 0 unavailable
StrategyType: RollingUpdate
MinReadySeconds: 0
RollingUpdateStrategy: 25% max unavailable, 25% max surge
Pod Template:
Labels: app=demo-nginx
Containers:
nginx:
Image: nginx
Port: <none>
Host Port: <none>
Environment: <none>
Mounts: <none>
Volumes: <none>
Conditions:
Type Status Reason
---- ------ ------
Available True MinimumReplicasAvailable
Progressing True NewReplicaSetAvailable
OldReplicaSets: <none>
NewReplicaSet: demo-nginx-645c888794 (1/1 replicas created)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ScalingReplicaSet 6m49s deployment-controller Scaled up replica set demo-nginx-645c888794 to 1
[root@k8s-master namespaces]# kubectl get replicaset
NAME DESIRED CURRENT READY AGE
demo-nginx-645c888794 1 1 1 5m14s
[root@k8s-master namespaces]# kubectl describe ReplicaSet demo-nginx-645c888794
Name: demo-nginx-645c888794
Namespace: default
Selector: app=demo-nginx,pod-template-hash=645c888794
Labels: app=demo-nginx
pod-template-hash=645c888794
Annotations: deployment.kubernetes.io/desired-replicas: 1
deployment.kubernetes.io/max-replicas: 2
deployment.kubernetes.io/revision: 1
Controlled By: Deployment/demo-nginx
Replicas: 1 current / 1 desired
Pods Status: 1 Running / 0 Waiting / 0 Succeeded / 0 Failed
Pod Template:
Labels: app=demo-nginx
pod-template-hash=645c888794
Containers:
nginx:
Image: nginx
Port: <none>
Host Port: <none>
Environment: <none>
Mounts: <none>
Volumes: <none>
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal SuccessfulCreate 9m1s replicaset-controller Created pod: demo-nginx-645c888794-5wxtb
[root@k8s-master namespaces]# kubectl get pods -o wide --namespace default
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
curl 1/1 Running 1 (3h35m ago) 3h38m 10.244.2.2 k8s-node2 <none> <none>
demo-nginx-645c888794-5wxtb 1/1 Running 0 12m 10.244.3.2 k8s-node3 <none> <none>
nginx 1/1 Running 0 3h13m 10.244.1.2 k8s-node1 <none> <none>
[root@k8s-master namespaces]# kubectl get all --namespace default
NAME READY STATUS RESTARTS AGE
pod/curl 1/1 Running 1 (3h43m ago) 3h46m
pod/demo-nginx-645c888794-5wxtb 1/1 Running 0 19m
pod/nginx 1/1 Running 0 3h20m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 4h56m
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/demo-nginx 1/1 1 1 19m
NAME DESIRED CURRENT READY AGE
replicaset.apps/demo-nginx-645c888794 1 1 1 19m
Context
[root@k8s-master namespaces]# kubectl config get-contexts
CURRENT NAME CLUSTER AUTHINFO NAMESPACE
* kubernetes-admin@kubernetes kubernetes kubernetes-admin
[root@k8s-master namespaces]# kubectl config set-context $(kubectl config current-context) --namespace=demo-namespace
Context "kubernetes-admin@kubernetes" modified.
[root@k8s-master namespaces]# kubectl config get-contexts
CURRENT NAME CLUSTER AUTHINFO NAMESPACE
* kubernetes-admin@kubernetes kubernetes kubernetes-admin demo-namespace
NameSpace
- kubectl get namespace
- kubectl get ns
NAME STATUS AGE
default Active 93m
kube-node-lease Active 93m
kube-public Active 93m
kube-system Active 93m
- kubectl describe namespace default
- kubectl create namespace test-nm
- kubectl apply -f demo-namespace.yaml
apiVersion: v1
kind: Namespace
metadata:
name: demo-namespace
- kubectl delete namespace test-nm
Deployment
- kubectl create deployment --image nginx demo-nginx
- kubectl get deployment
NAME READY UP-TO-DATE AVAILABLE AGE
demo-nginx 1/1 1 1 82s
[root@k8s-master namespaces]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
curl 1/1 Running 1 (3h24m ago) 3h27m 10.244.2.2 k8s-node2
demo-nginx-645c888794-5wxtb 1/1 Running 0 50s 10.244.3.2 k8s-node3
nginx 1/1 Running 0 3h1m 10.244.1.2 k8s-node1
- 获取Pod列表
kubectl get pods --all-namespaces
运行Curl Pod
- Run Curl Pod
kubectl run curl --image=radial/busyboxplus:curl -it
[root@k8s-master manifests]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
curl 1/1 Running 1 (11m ago) 14m 10.244.2.2 k8s-node2 <none> <none>
kubectl get pod curl -o wide
kubectl describe pod curl
kubectl exec -it curl -- /bin/sh
#假如当前POD（curl为pod name）只有一个容器,运行以下命令即可
#如果POD内有多个container 通过 --container container-name 进行选择
# 默认的namespace为， 可以通过 -n namespace 指定用户自己的namespace
[root@k8s-master k8s]# kubectl get all --namespace kubernetes-dashboard
NAME READY STATUS RESTARTS AGE
pod/dashboard-metrics-scraper-856586f554-ksvfj 1/1 Running 0 17h
pod/kubernetes-dashboard-78c79f97b4-4gtl2 1/1 Running 0 17h
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/dashboard-metrics-scraper ClusterIP 10.98.63.120 <none> 8000/TCP 17h
service/kubernetes-dashboard ClusterIP 10.110.75.181 <none> 443/TCP 17h
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/dashboard-metrics-scraper 1/1 1 1 17h
deployment.apps/kubernetes-dashboard 1/1 1 1 17h
NAME DESIRED CURRENT READY AGE
replicaset.apps/dashboard-metrics-scraper-856586f554 1 1 1 17h
replicaset.apps/kubernetes-dashboard-78c79f97b4 1 1 1 17h
[root@k8s-master k8s]# kubectl get pod --namespace kubernetes-dashboard
NAME READY STATUS RESTARTS AGE
dashboard-metrics-scraper-856586f554-ksvfj 1/1 Running 0 17h
kubernetes-dashboard-78c79f97b4-4gtl2 1/1 Running 0 17h
[root@k8s-master k8s]# kubectl get pod -o wide --namespace kubernetes-dashboard
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
dashboard-metrics-scraper-856586f554-ksvfj 1/1 Running 0 17h 10.244.2.3 k8s-node2 <none> <none>
kubernetes-dashboard-78c79f97b4-4gtl2 1/1 Running 0 17h 10.244.1.3 k8s-node1 <none> <none>