kubeadm deploy kubernetes
Introduction to kubeadm deployment method
1. Use kubeadm
tools to quickly deploy kubernetes
clusters:
1. Create a master node
kubeadm init
2. Add node nodes to the clusterkubeadm join <master的IP和PORT>
2. Kubernetes cluster machine requirements
Operating system CentOS7.x-86_x64;
hardware configuration: 2GB+ RAM, 2+ CPU, hard disk 30GB+;
network communication between all machines in the cluster;
all machines in the cluster can access the external network (need to pull the image);
prohibit swap partition;
3. The ultimate goal
1) Install docker and kubeadm on all nodes;
2) Deploy the kubernetes master;
3) Deploy the container network plug-in;
4) Deploy the kubernetes node and add the node to the kubernetes cluster;
5) Deploy the dashboard web page to visually view the kubernetes resources;
Single master node deployment planning
Multi-master node deployment plan
Single master node environment preparation
1. k8s cluster host role IP
k8s-master 192.168.6.112
k8s-worker1 192.168.6.113
k8s-worker2 192.168.6.114
2. Configure the network
vim /etc/sysconfig/network-scripts/ifcfg-ens32
===============================
TYPE="Ethernet"
BOOTPROTO="static"
DEVICE="ens32"
ONBOOT="yes"
# 分别配置三台服务器的ip地址
IPADDR="192.168.6.112"
# IPADDR="192.168.6.113"
# IPADDR="192.168.6.114"
GATEWAY="192.168.6.1"
NETMASK="255.255.255.0"
DNS1="8.8.8.8"
===============================
service network restart
3. Turn off the firewall
systemctl stop firewalld
systemctl disable firewalld
4. Close selinux
# 重启永久关闭selinux
sed -i 's/SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config
# 当前环境临时关闭selinux
setenforce 0
5. Close the swap partition
# 临时开启 swapon -a
# 临时关闭swap
swapoff -a
# 永久关闭swap
sed -ri 's/.*swap.*/#&/' /etc/fstab
=====================
UUID=a6f57ca4-f414-43ee-a102-39e970fe4741 swap swap defaults 0 0
6. Modify the host name
# 192.168.6.112
hostnamectl set-hostname k8s-master
# 192.168.6.113
hostnamectl set-hostname k8s-worker1
# 192.168.6.114
hostnamectl set-hostname k8s-worker2
7. Add local dns to the /etc/hosts file
# 追加写入文件内容
cat >> /etc/hosts << EOF
192.168.6.112 k8s-master
192.168.6.113 k8s-worker1
192.168.6.114 k8s-worker2
EOF
8. Pass the bridged IPv4
traffic to the iptables chain
# 覆盖写入文件内容
cat > /etc/sysctl.d/k8s.conf << EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
# 以上配置立即生效
sysctl --system
9. Time synchronization
# 1、安装ntpdate
yum install ntpdate -y
# 2、手动同步时间
# ntpdate time.windows.com
ntpdate ntp1.aliyun.com
# 3、定时同步时间任务
echo " */3 * * * * /usr/sbin/ntpdate -u ntp1.aliyun.com > /dev/null 2>&1 " \
>> /var/spool/cron/root
# 4、使用date命令查看当前时间
date
# 5、系统时间同步到硬件,防止系统重启后时间被还原
# -w,--systohc; set the hardware clock from the current system time
hwclock --systohc
7. Install docker/kubeadm/kubelet on all nodes.
The default CRI (container runtime) of Kubernetes is docker, so install docker first.
(1) Install Docker
wget https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo -O /etc/yum.repos.d/docker-ce.repo
yum -y install docker-ce-18.06.1.ce-3.el7
systemctl enable docker && systemctl start docker
docker --version
(2) Add Alibaba Cloud YUM software source
and set warehouse address
cat > /etc/docker/daemon.json << EOF
{
"registry-mirrors": ["https://0s2uk8va.mirror.aliyuncs.com"]
}
EOF
systemctl daemon-reload
systemctl restart docker
docker info
add yum source
cat > /etc/yum.repos.d/kubernetes.repo << EOF
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
(3) Install kubeadm, kubelet and kubectl
yum install -y kubelet-1.18.0 kubeadm-1.18.0 kubectl-1.18.0
systemctl enable kubelet
8. Deploy Kubernetes Master
(1) Execute at 192.168.6.112 (Master)
kubeadm init \
--apiserver-advertise-address=192.168.6.112 \
--image-repository registry.aliyuncs.com/google_containers \
--kubernetes-version v1.18.0 \
--service-cidr=10.96.0.0/12 \
--pod-network-cidr=10.244.0.0/16
Since the default pull mirror address k8s.gcr.io cannot be accessed in China, the address of the Alibaba Cloud mirror warehouse is specified here.
The result after execution:
Your Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/
Then you can join any number of worker nodes by running the following on each as root:
kubeadm join 192.168.6.112:6443 --token 2be5c9.y7fini11lk3f7f3f \
--discovery-token-ca-cert-hash sha256:bf1c7f2fab37bf8f6d248087e19c2ada17327bdffc95159d0ec3c2bf8d14973a
docker images
=======================
REPOSITORY TAG IMAGE ID CREATED SIZE
registry.aliyuncs.com/google_containers/kube-proxy v1.18.0 43940c34f24f 2 years ago 117MB
registry.aliyuncs.com/google_containers/kube-apiserver v1.18.0 74060cea7f70 2 years ago 173MB
registry.aliyuncs.com/google_containers/kube-controller-manager v1.18.0 d3e55153f52f 2 years ago 162MB
registry.aliyuncs.com/google_containers/kube-scheduler v1.18.0 a31f78c7c8ce 2 years ago 95.3MB
registry.aliyuncs.com/google_containers/pause 3.2 80d28bedfe5d 2 years ago 683kB
registry.aliyuncs.com/google_containers/coredns 1.6.7 67da37a9a360 2 years ago 43.8MB
registry.aliyuncs.com/google_containers/etcd 3.4.3-0 303ce5db0e90 2 years ago 288MB
(2) Use the kubectl tool:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
Check
kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master NotReady master 4m39s v1.18.0
9. Join Kubernetes Node
(1) Execute on other nodes 192.168.31.62/63 (Node) To
add a new node to the cluster, execute the kubeadm join command output by kubeadm init:
kubeadm join 192.168.6.112:6443 --token 2be5c9.y7fini11lk3f7f3f \
--discovery-token-ca-cert-hash sha256:bf1c7f2fab37bf8f6d248087e19c2ada17327bdffc95159d0ec3c2bf8d14973a
kubectl get nodes
========================
NAME STATUS ROLES AGE VERSION
k8s-master NotReady master 8m34s v1.18.0
k8s-worker1 NotReady <none> 13s v1.18.0
k8s-worker2 NotReady <none> 8s v1.18.0
The default token is valid for 24 hours. At this time, the token needs to be recreated. The operation is as follows:
kubeadm token create --print-join-command
10. The default image address of the Pod network plug-in (CNI) installed on the master node
cannot be accessed, and the sed command is changed to the docker hub image warehouse.
wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
Make sure you can access the quay.io register. If the Pod mirror download fails, you can change the mirror address
kubectl get pods -n kube-system
=====================================
NAME READY STATUS RESTARTS AGE
coredns-7ff77c879f-gzg2l 1/1 Running 0 16m
coredns-7ff77c879f-tg9lr 1/1 Running 0 16m
etcd-k8s-master 1/1 Running 0 17m
kube-apiserver-k8s-master 1/1 Running 0 17m
kube-controller-manager-k8s-master 1/1 Running 0 17m
kube-flannel-ds-gh68f 1/1 Running 0 113s
kube-flannel-ds-mm7bd 1/1 Running 0 113s
kube-flannel-ds-rj254 1/1 Running 0 113s
kube-proxy-4hjwr 1/1 Running 0 8m40s
kube-proxy-ddtdq 1/1 Running 0 8m45s
kube-proxy-ghxxc 1/1 Running 0 16m
kube-scheduler-k8s-master 1/1 Running 0 17m
11. Test the kubernetes cluster
Create a pod in the Kubernetes cluster to verify whether it is running normally:
kubectl create deployment nginx --image=nginx
kubectl expose deployment nginx --port=80 --type=NodePort
kubectl get pod,svc
NAME READY STATUS RESTARTS AGE
pod/nginx-f89759699-48qvm 0/1 ContainerCreating 0 27s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 19m
service/nginx NodePort 10.97.34.213 <none> 80:31499/TCP 7s
address:
http://192.168.6.112:31499
http://192.168.6.113:31499
http://192.168.6.114:31499
Uninstall kubernetes
kubeadm reset -f
modprobe -r ipip
lsmod
rm -rf ~/.kube/
rm -rf /etc/kubernetes/
rm -rf /etc/systemd/system/kubelet.service.d
rm -rf /etc/systemd/system/kubelet.service
rm -rf /usr/bin/kube*
rm -rf /etc/cni
rm -rf /opt/cni
rm -rf /var/lib/etcd
rm -rf /var/etcd
yum clean all
yum remove kube*
kubeadm init command documentation
kubeadm init --help
===========================
Run this command in order to set up the Kubernetes control plane
The "init" command executes the following phases:
`
preflight Run pre-flight checks
kubelet-start Write kubelet settings and (re)start the kubelet
certs Certificate generation
/ca Generate the self-signed Kubernetes CA to provision identities for other Kubernetes components
/apiserver Generate the certificate for serving the Kubernetes API
/apiserver-kubelet-client Generate the certificate for the API server to connect to kubelet
/front-proxy-ca Generate the self-signed CA to provision identities for front proxy
/front-proxy-client Generate the certificate for the front proxy client
/etcd-ca Generate the self-signed CA to provision identities for etcd
/etcd-server Generate the certificate for serving etcd
/etcd-peer Generate the certificate for etcd nodes to communicate with each other
/etcd-healthcheck-client Generate the certificate for liveness probes to healthcheck etcd
/apiserver-etcd-client Generate the certificate the apiserver uses to access etcd
/sa Generate a private key for signing service account tokens along with its public key
kubeconfig Generate all kubeconfig files necessary to establish the control plane and the admin kubeconfig file
/admin Generate a kubeconfig file for the admin to use and for kubeadm itself
/kubelet Generate a kubeconfig file for the kubelet to use *only* for cluster bootstrapping purposes
/controller-manager Generate a kubeconfig file for the controller manager to use
/scheduler Generate a kubeconfig file for the scheduler to use
control-plane Generate all static Pod manifest files necessary to establish the control plane
/apiserver Generates the kube-apiserver static Pod manifest
/controller-manager Generates the kube-controller-manager static Pod manifest
/scheduler Generates the kube-scheduler static Pod manifest
etcd Generate static Pod manifest file for local etcd
/local Generate the static Pod manifest file for a local, single-node local etcd instance
upload-config Upload the kubeadm and kubelet configuration to a ConfigMap
/kubeadm Upload the kubeadm ClusterConfiguration to a ConfigMap
/kubelet Upload the kubelet component config to a ConfigMap
upload-certs Upload certificates to kubeadm-certs
mark-control-plane Mark a node as a control-plane
bootstrap-token Generates bootstrap tokens used to join a node to a cluster
kubelet-finalize Updates settings relevant to the kubelet after TLS bootstrap
/experimental-cert-rotation Enable kubelet client certificate rotation
addon Install required addons for passing Conformance tests
/coredns Install the CoreDNS addon to a Kubernetes cluster
/kube-proxy Install the kube-proxy addon to a Kubernetes cluster
`
Usage:
kubeadm init [flags]
kubeadm init [command]
Available Commands:
phase Use this command to invoke single phase of the init workflow
Flags:
--apiserver-advertise-address string The IP address the API Server will advertise it‘s listening on. If not set the default network interface will be used.
--apiserver-bind-port int32 Port for the API Server to bind to. (default 6443)
--apiserver-cert-extra-sans strings Optional extra Subject Alternative Names (SANs) to use for the API Server serving certificate. Can be both IP addresses and DNS names.
--cert-dir string The path where to save and store the certificates. (default "/etc/kubernetes/pki")
--certificate-key string Key used to encrypt the control-plane certificates in the kubeadm-certs Secret.
--config string Path to a kubeadm configuration file.
--control-plane-endpoint string Specify a stable IP address or DNS name for the control plane.
--cri-socket string Path to the CRI socket to connect. If empty kubeadm will try to auto-detect this value; use this option only if you have more than one CRI installed or if you have non-standard CRI socket.
--dry-run Don‘t apply any changes; just output what would be done.
-k, --experimental-kustomize string The path where kustomize patches for static pod manifests are stored.
--feature-gates string A set of key=value pairs that describe feature gates for various features. Options are:
IPv6DualStack=true|false (ALPHA - default=false)
PublicKeysECDSA=true|false (ALPHA - default=false)
-h, --help help for init
--ignore-preflight-errors strings A list of checks whose errors will be shown as warnings. Example: 'IsPrivilegedUser,Swap'. Value 'all' ignores errors from all checks.
--image-repository string Choose a container registry to pull control plane images from (default "k8s.gcr.io")
--kubernetes-version string Choose a specific Kubernetes version for the control plane. (default "stable-1")
--node-name string Specify the node name.
--pod-network-cidr string Specify range of IP addresses for the pod network. If set, the control plane will automatically allocate CIDRs for every node.
--service-cidr string Use alternative range of IP address for service VIPs. (default "10.96.0.0/12")
--service-dns-domain string Use alternative domain for services, e.g. "myorg.internal". (default "cluster.local")
--skip-certificate-key-print Don‘t print the key used to encrypt the control-plane certificates.
--skip-phases strings List of phases to be skipped
--skip-token-print Skip printing of the default bootstrap token generated by 'kubeadm init'.
--token string The token to use for establishing bidirectional trust between nodes and control-plane nodes. The format is [a-z0-9]{
6}\.[a-z0-9]{
16} - e.g. abcdef.0123456789abcdef
--token-ttl duration The duration before the token is automatically deleted (e.g. 1s, 2m, 3h). If set to '0', the token will never expire (default 24h0m0s)
--upload-certs Upload control-plane certificates to the kubeadm-certs Secret.
Global Flags:
--add-dir-header If true, adds the file directory to the header
--log-file string If non-empty, use this log file
--log-file-max-size uint Defines the maximum size a log file can grow to. Unit is megabytes. If the value is 0, the maximum file size is unlimited. (default 1800)
--rootfs string [EXPERIMENTAL] The path to the 'real' host root filesystem.
--skip-headers If true, avoid header prefixes in the log messages
--skip-log-headers If true, avoid headers when opening log files
-v, --v Level number for the log level verbosity
Use "kubeadm init [command] --help" for more information about a command.
——Shang Silicon Valley k8s tutorial study notes