Article Directory
1.5, deploy kube-nginx
-
master
Node needskube-nginx
-
Use Nginx 4-layer transparent proxy function to achieve high availability of k8s nodes (master nodes and nginx nodes) to access kube-apiserver
-
The kube-controller-manager and kube-scheduler of the control node are multi-instance deployments, so as long as one instance is normal, the cluster can be highly available
-
Pods in the cluster use k8s service domain name kubernetes to access kube-apiserver, kube-dns will automatically resolve the IP of multiple kube-apiserver nodes, so it is also highly available
-
In each Nginx process, the backend is connected to multiple apiserver instances, and Nginx performs health checks and load balancing on them
1.5.0, download nginx binary file
k8s-01:~ # cd /opt/k8s/packages/
k8s-01:/opt/k8s/packages # wget http://nginx.org/download/nginx-1.16.1.tar.gz
k8s-01:/opt/k8s/packages # tar xf nginx-1.16.1.tar.gz
1.5.1, compile and deploy nginx
k8s-01:~ # cd /opt/k8s/packages/nginx-1.16.1/
k8s-01:/opt/k8s/packages/nginx-1.16.1 # ./configure --prefix=$(pwd)/nginx-prefix \
--with-stream \
--without-http \
--without-http_uwsgi_module && \
make && \
make install
--with-stream
Enable Layer 4 transparent forwarding (TCP Proxy) function--without-xxx
Turn off the function, so that the generated dynamic link binary program depends on the least
1.5.2, configure nginx.conf
k8s-01:~ # cd /opt/k8s/conf/
k8s-01:/opt/k8s/conf # cat > kube-nginx.conf <<EOF
worker_processes 1;
events {
worker_connections 1024;
}
stream {
upstream backend {
hash \$remote_addr consistent;
server 192.168.72.39:6443 max_fails=3 fail_timeout=30s;
server 192.168.72.40:6443 max_fails=3 fail_timeout=30s;
server 192.168.72.41:6443 max_fails=3 fail_timeout=30s;
}
server {
listen *:8443;
proxy_connect_timeout 1s;
proxy_pass backend;
}
}
EOF
- Note: The ip address to
master节点
theip
,以自己的环境为准
1.5.3, configure nginx as systemctl management
k8s-01:~ # cd /opt/k8s/conf/
k8s-01:/opt/k8s/conf # cat > kube-nginx.service <<EOF
[Unit]
Description=kube-apiserver nginx proxy
After=network.target
After=network-online.target
Wants=network-online.target
[Service]
Type=forking
ExecStartPre=/opt/k8s/server/kube-nginx/sbin/nginx \
-c /opt/k8s/server/kube-nginx/conf/kube-nginx.conf \
-p /opt/k8s/server/kube-nginx -t
ExecStart=/opt/k8s/server/kube-nginx/sbin/nginx \
-c /opt/k8s/server/kube-nginx/conf/kube-nginx.conf \
-p /opt/k8s/server/kube-nginx
ExecReload=/opt/k8s/server/kube-nginx/sbin/nginx \
-c /opt/k8s/server/kube-nginx/conf/kube-nginx.conf \
-p /opt/k8s/server/kube-nginx -s reload
PrivateTmp=true
Restart=always
RestartSec=5
StartLimitInterval=0
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
EOF
1.5.4, distribute nginx binary files and configuration files
#!/usr/bin/env bash
source /opt/k8s/bin/k8s-env.sh
for host in ${MASTER_IPS[@]}
do
printf "\e[1;34m${host}\e[0m\n"
ssh root@${host} "mkdir -p /opt/k8s/server/kube-nginx/{conf,logs,sbin}"
scp /opt/k8s/packages/nginx-1.16.1/nginx-prefix/sbin/nginx ${host}:/opt/k8s/server/kube-nginx/sbin/
scp /opt/k8s/conf/kube-nginx.conf ${host}:/opt/k8s/server/kube-nginx/conf/
scp /opt/k8s/conf/kube-nginx.service ${host}:/etc/systemd/system/
done
1.5.5, start kube-nginx service
#!/usr/bin/env bash
source /opt/k8s/bin/k8s-env.sh
for host in ${MASTER_IPS[@]}
do
printf "\e[1;34m${host}\e[0m\n"
ssh root@${host} "systemctl daemon-reload && \
systemctl enable kube-nginx --now && \
systemctl status kube-nginx | grep Active"
done