官方提供的几种Kubernetes部署方式
minikube Minikube是一个工具，可以在本地快速运行一个单点的Kubernetes，尝试Kubernetes或日常开发的用户使用。不能用于生产环境。 kubeadm Kubeadm也是一个工具，提供kubeadm init和kubeadm join指令，用于快速部署Kubernetes集群。 二进制包 从官方下载发行版的二进制包，手动部署每个组件，组成Kubernetes集群。 小结：生产环境中部署Kubernetes集群，只有Kubeadm和二进制包可选，Kubeadm降低部署门槛，但屏蔽了很多细节，遇到问题很难排查。我们这里使用二进制包部署Kubernetes集群，我也是推荐大家使用这种方式，虽然手动部署麻烦点，但学习很多工作原理，更有利于后期维护。
软件环境：
| 软件 | 版本 |
| 操作系统 | CentOS7.5_x64 |
| Docker | 18-ce |
| Kubernetes | 1.12 |
服务器角色：
| 角色 |
IP |
组件 |
| k8s-master01 | 192.168.125.122 | kube-apiserver，kube-controller-manager，kube-scheduler，etcd |
| k8s-node01 | 192.168.125.123 | kubelet，kube-proxy，docker，flannel，etcd |
| k8s-node02 | 192.168.125.124 | kubelet，kube-proxy，docker，flannel，etcd |
基础环境配置：
k8s-master01：
hostnamectl set-hostname k8s-master01 echo "192.168.125.122 k8s-master01" >> /etc/hosts systemctl stop firewalld;systemctl disable firewalld setenforce 0;sed -s '/SELINUX/s/enforcing/disabled/g' /etc/selinux/config yum install ntpdate wget unzip lrzsz vim -y;ntpdate -u cn.ntp.org.cn swapoff -a cat > /etc/sysctl.d/k8s.conf << EOF net.bridge.bridge-nf-call-ip6tables = 1 net.bridge.bridge-nf-call-iptables = 1 EOF sysctl --system logout
k8s-node01：
hostnamectl set-hostname k8s-node01 echo "192.168.125.123 k8s-node01" >> /etc/hosts systemctl stop firewalld;systemctl disable firewalld setenforce 0;sed -s '/SELINUX/s/enforcing/disabled/g' /etc/selinux/config yum install ntpdate wget unzip lrzsz vim -y;ntpdate -u cn.ntp.org.cn swapoff -a cat > /etc/sysctl.d/k8s.conf << EOF net.bridge.bridge-nf-call-ip6tables = 1 net.bridge.bridge-nf-call-iptables = 1 EOF sysctl --system logout
k8s-node02：
hostnamectl set-hostname k8s-node02 echo "192.168.125.124 k8s-node02" >> /etc/hosts systemctl stop firewalld;systemctl disable firewalld setenforce 0;sed -s '/SELINUX/s/enforcing/disabled/g' /etc/selinux/config yum install ntpdate wget unzip lrzsz vim -y;ntpdate -u cn.ntp.org.cn swapoff -a cat > /etc/sysctl.d/k8s.conf << EOF net.bridge.bridge-nf-call-ip6tables = 1 net.bridge.bridge-nf-call-iptables = 1 EOF sysctl --system logout
1. 部署Etcd集群
使用cfssl来生成自签证书，先下载cfssl工具： wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64 chmod +x cfssl_linux-amd64 cfssljson_linux-amd64 cfssl-certinfo_linux-amd64 mv cfssl_linux-amd64 /usr/local/bin/cfssl mv cfssljson_linux-amd64 /usr/local/bin/cfssljson mv cfssl-certinfo_linux-amd64 /usr/bin/cfssl-certinfo 注：可以上传脚本cfssl.sh并执行。
1.1 生成证书创建
以下三个文件：
cat > ca-config.json <<EOF
{
"signing": {
"default": {
"expiry": "87600h"
},
"profiles": {
"www": {
"expiry": "87600h",
"usages": [
"signing",
"key encipherment",
"server auth",
"client auth"
]
}
}
}
}
EOF
cat > ca-csr.json <<EOF
{
"CN": "etcd CA",
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"L": "Beijing",
"ST": "Beijing"
}
]
}
EOF
#-----------------------
cat > server-csr.json <<EOF
{
"CN": "etcd",
"hosts": [
"192.168.125.122",
"192.168.125.123",
"192.168.125.124"
],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"L": "BeiJing",
"ST": "BeiJing"
}
]
}
EOF
生成证书：
cfssl gencert -initca ca-csr.json | cfssljson -bare ca -
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=www server-csr.json | cfssljson -bare server
# ls *pemca-key.pem ca.pem server-key.pem server.pem
证书这块知道怎么生成、怎么用即可，建议暂时不必过多研究。
脚本配置：
创建目录/root/k8s/etcd-cert，并将文件etcd-cert.sh 、etcd.sh上传到该目录下，将文件etcd-cert.sh中的hosts做相应的更改，然后执行bash bash /root/k8s/etcd-cert/etcd-cert.sh即可完成证书的申请。
1.2 部署Etcd
二进制包下载地址：https://github.com/coreos/etcd/releases/tag/v3.3.10
以下部署步骤在规划的三个etcd节点操作一样，唯一不同的是etcd配置文件中的服务器IP要写当前的：
解压二进制包：：
# mkdir /opt/etcd/{bin,cfg,ssl} -p
# tar zxvf etcd-v3.3.10-linux-amd64.tar.gz
# mv etcd-v3.3.10-linux-amd64/{etcd,etcdctl} /opt/etcd/bin
创建etcd配置文件：
# cat /opt/etcd/cfg/etcd
#[Member]
ETCD_NAME="etcd01"
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="https://192.168.125.122:2380"
ETCD_LISTEN_CLIENT_URLS="https://192.168.125.122:2379"
#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.125.122:2380"
ETCD_ADVERTISE_CLIENT_URLS="https://192.168.125.122:2379"
ETCD_INITIAL_CLUSTER="etcd01=https://192.168.125.122:2380,etcd02=https://192.168.125.123:2380,etcd03=https://192.168.125.124:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"
ETCD_NAME 节点名称ETCD_DATA_DIR 数据目录
ETCD_LISTEN_PEER_URLS 集群通信监听地址
ETCD_LISTEN_CLIENT_URLS 客户端访问监听地址
ETCD_INITIAL_ADVERTISE_PEER_URLS 集群通告地址
ETCD_ADVERTISE_CLIENT_URLS 客户端通告地址
ETCD_INITIAL_CLUSTER 集群节点地址
ETCD_INITIAL_CLUSTER_TOKEN 集群
TokenETCD_INITIAL_CLUSTER_STATE 加入集群的当前状态，new是新集群，existing表示加入已有集群
systemd管理etcd：
# cat /usr/lib/systemd/system/etcd.service
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target
[Service]
Type=notify
EnvironmentFile=/opt/etcd/cfg/etcd
ExecStart=/opt/etcd/bin/etcd --name=${ETCD_NAME} --data-dir=${ETCD_DATA_DIR} --listen-peer-urls=${ETCD_LISTEN_PEER_URLS} --listen-client-urls=${ETCD_LISTEN_CLIENT_URLS},http://127.0.0.1:2379 --advertise-client-urls=${ETCD_ADVERTISE_CLIENT_URLS} --initial-advertise-peer-urls=${ETCD_INITIAL_ADVERTISE_PEER_URLS} --initial-cluster=${ETCD_INITIAL_CLUSTER} --initial-cluster-token=${ETCD_INITIAL_CLUSTER_TOKEN} --initial-cluster-state=new --cert-file=/opt/etcd/ssl/server.pem --key-file=/opt/etcd/ssl/server-key.pem --peer-cert-file=/opt/etcd/ssl/server.pem --peer-key-file=/opt/etcd/ssl/server-key.pem --trusted-ca-file=/opt/etcd/ssl/ca.pem --peer-trusted-ca-file=/opt/etcd/ssl/ca.pem
Restart=on-failure
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
把刚才生成的证书拷贝到配置文件中的位置：
# cp ca*pem server*pem /opt/etcd/ssl
启动并设置开启启动：
# systemctl start etcd
# systemctl enable etcd
其它节点配置同上
配置k8s-master到k8s-node节点的ssh认证
ssh免秘钥：
ssh-keygen
ssh-copy-id [email protected]
ssh-copy-id [email protected]
scp -r /opt/etcd/ [email protected]:/opt/
scp /usr/lib/systemd/system/etcd.service [email protected]:/usr/lib/systemd/system/
scp /usr/lib/systemd/system/etcd.service [email protected]:/usr/lib/systemd/system/
scp -r /opt/etcd/ [email protected]:/opt/
scp /usr/lib/systemd/system/etcd.service [email protected]:/usr/lib/systemd/system/
scp /usr/lib/systemd/system/etcd.service [email protected]:/usr/lib/systemd/system/
修改k8s-node节点的/opt/etcd/cfg/etcd 配置文件，监听IP更改为本机。
ssh [email protected] "systemctl start etcd;systemctl enable etcd"
ssh [email protected] "systemctl start etcd;systemctl enable etcd"
都部署完成后，检查etcd集群状态：
# /opt/etcd/bin/etcdctl \
--ca-file=ca.pem --cert-file=server.pem --key-file=server-key.pem \
--endpoints="https://192.168.125.122:2379,https://192.168.125.123:2379,https://192.168.125.124:2379" \
cluster-health
执行结果：
# /opt/etcd/bin/etcdctl \
> --ca-file=ca.pem --cert-file=server.pem --key-file=server-key.pem \
> --endpoints="https://192.168.125.122:2379,https://192.168.125.123:2379,https://192.168.125.124:2379" \
> cluster-health
member 1fd18cc9663e3095 is healthy: got healthy result from https://192.168.125.123:2379
member 3a7ee6dbc6ae9ddb is healthy: got healthy result from https://192.168.125.124:2379
member 79714bdc6157cf6d is healthy: got healthy result from https://192.168.125.122:2379
cluster is healthy
或者：给脚本etcd.sh 执行权限，然后运行指令
./etcd.sh etcd01 192.168.125.122 etcd02=https://192.168.125.123:2380,etcd03=https://192.168.125.124:2380
注：第一个启动的节点，会有报错忽略，先去启动其它节点的etcd服务即可。
如果输出上面信息，就说明集群部署成功。如果有问题第一步先看日志：/var/log/message 或 journalctl -u etcd