Cloud security attack and defense (12) manual construction of K8S environment construction

Manually build K8S environment build

First, we prepared three Centos7 machines in the early stage, and the configuration is as follows:

CPU name	IP	system version
k8s-master	192.168.41.141	Centos7
k8s-node1	192.168.41.142	Centos7
k8s-node2	192.168.41.143	Centos7

Preparation

First execute the following commands on all three machines

# 关闭防火墙
systemctl stop firewalld
systemctl disable firewalld
# 永久关闭 selinux
sed -i 's/enforcing/disabled/' /etc/selinux/config
# 永久关闭 swap
sed -ri 's/.*swap.*/#&/' /etc/fstab
# 修改/etc/hosts
vim /etc/hosts
192.168.41.141 k8s-master
192.168.41.142 k8s-node1
192.168.41.143 k8s-node2
# 将桥接的IPv4流量传递到iptables的链
cat > /etc/sysctl.d/k8s.conf << EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
# 生效
sysctl --system  
# 时间同步
yum install ntpdate -y
ntpdate time.windows.com

Then configure Alibaba Cloud's K8s source on the three machines and execute the following command

cat > /etc/yum.repos.d/kubernetes.repo << EOF
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

Then install docker, kubeadm, kubelet, kubectl on all three machines

To install docker, execute the following commands on all three machines to install with one click

curl -s https://get.docker.com/ | sh

The installation is complete as follows:

To configure the docker image download accelerator, execute the following command:

vim /etc/docker/daemon.json

{
    
    
  "registry-mirrors": ["https://b9pmyelo.mirror.aliyuncs.com"],
  "exec-opts": ["native.cgroupdriver=systemd"]
}
 #查看docker信息，进行确认
systemctl restart docker
docker info  

The following message appears, indicating that the configuration is complete

Then execute the following command to install kubelet, kubeadm and kubectl

#安装 kubelet、kubeadm 和 kubectl
yum install -y kubelet-1.21.0 kubeadm-1.21.0 kubectl-1.21.0
#设置 kubelet 开机自启
systemctl enable kubelet

Deploy K8S-master

Execute the following command on the master node to initialize the master

kubeadm init --apiserver-advertise-address=192.168.41.141 --image-repository registry.aliyuncs.com/google_containers --kubernetes-version v1.21.0 --service-cidr=10.96.0.0/12 --pod-network-cidr=10.244.0.0/16 --ignore-preflight-errors=all

--apiserver-advertise-address 集群通告地址
--image-repository 由于默认拉取镜像地址k8s.gcr.io国内无法访问，这里指定阿里云镜像仓库地址
--kubernetes-version K8s版本，与上面安装的一致
--service-cidr 集群内部虚拟网络，Pod统一访问入口
--pod-network-cidr Pod网络，与下面部署的CNI网络组件yaml中保持一致

Or bootstrap with a config file:

vi kubeadm.conf
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
kubernetesVersion: v1.21.0
imageRepository: registry.aliyuncs.com/google_containers 
networking:
  podSubnet: 10.244.0.0/16 
  serviceSubnet: 10.96.0.0/12 

Then execute the following command to initialize

kubeadm init --config kubeadm.conf --ignore-preflight-errors=all 

After the initialization is complete, a join command will be output at the end. Remember first, use the following

Then copy the connection k8s authentication file used by kubectl to the default path:

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Deploy K8S-node

Execute the following command on two node nodes (192.168.41.142/143) to join the K8S cluster

kubeadm join 192.168.41.141:6443 --token sec4pk.nnihf0tismgn6kax --discovery-token-ca-cert-hash sha256:a8a0adf8b5fd9adb6ac8a2977456bd1671055146ed5711eaab5280d6541986fd 

The default token is valid for 24 hours, and when it expires, the token is no longer available. At this time, the token needs to be recreated, which can be generated directly by using the command:

kubeadm token create --print-join-command

Deploy Container Networking (CNI)

Deployment network Calico is a pure three-tier data center network solution, which is currently the mainstream network solution for Kubernetes. Execute the following command to download YAML:

wget https://docs.projectcalico.org/v3.19/manifests/calico.yaml --no-check-certificate

After downloading, you need to modify the definition of the Pod network (CALICO_IPV4POOL_CIDR), which
is the same as specified by the --pod-network-cidr of kubeadm init

vim calico.yaml

By default, the images used in calico.yaml all come from the docker.io foreign image source, here we can delete the docker.io prefix to make the image download from the domestic image acceleration site

cat calico.yaml |grep 'image:'
sed -i 's#docker.io/##g' calico.yaml

After modifying the file, deploy:

#部署
kubectl apply -f calico.yaml
#查看状态，执行完上一条命令需要等一会才全部 running
kubectl get pods -n kube-system

Wait until the Calico Pods are running, and the nodes will be ready

Deploy Dashboard

Dashboard is an official Web UI that can be used for basic management of K8s resources. Execute the following command to download the yaml file. By default, the Dashboard can only be accessed within the cluster. Modify the Service to NodePort type and expose it to the outside:

wget https://github.com/kubernetes/dashboard/releases/tag/v2.4.0/aio/deploy/recommended.yaml

Modify as follows, the port range of nodePort is 30000-32767, here is set to 31000, and add type: NodePort

vim recommended.yaml

Execute the following command application

kubectl apply -f recommended.yaml

Create a service account on the master node and bind the default cluster-admin administrator cluster role, execute the following command

# 创建用户
kubectl create serviceaccount dashboard-admin -n kube-system
# 用户授权
kubectl create clusterrolebinding dashboard-admin --clusterrole=cluster-admin -
-serviceaccount=kube-system:dashboard-admin
# 获取用户 Token
kubectl describe secrets -n kube-system $(kubectl -n kube-system get secret |
awk '/dashboard-admin/{print $1}')

Then we get the following token, and then use the output token to log in to the Dashboard.

eyJhbGciOiJSUzI1NiIsImtpZCI6IjNpbzFJbTg4UjlpcjFBdS1rb1J3NzFtY3BETlhtVkQ3S0hXZWwwU0MwN1UifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkYXNoYm9hcmQtYWRtaW4tdG9rZW4tYzdzcHAiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiZGFzaGJvYXJkLWFkbWluIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiZjAyMTE1OWMtZjcyMC00YTZlLWFiY2MtYzIzYWRhZjBiZjk5Iiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Omt1YmUtc3lzdGVtOmRhc2hib2FyZC1hZG1pbiJ9.aD_ZtS0domXxtWz-2BaGmZebJMqoNvWqHgJ4K7kQ9eir5JvIqTsrxM14dNrUrEFRZC2hw6Gn_xz7Nezy81XPU64HHcbGiNvNU8K7OwvTWwTOpDBRPho1CaxJsKBrlQwoNf1pzoShqO-JdL4kVfJUmKthjUqv8QduwVzEkCWeTXgcHOoPnsOaJSXJzwanAC4e5pIovcjMGQJU4W87T8uVW4bdO4w48c-101-mMYHMouKVRxF8OOTuGHFXUDpYCKAOvfA73gtwoyi_4wiSqS7NSZZTGwFfppUYDomjoA3FUFubZ2xLoc8fN2GoFzzTcylxFHTCfupJM2nUVs9vxQbJw

Then our browser visits the URL: https://master:31000 or https://node:31000

Enter the obtained token to complete the login