A docker and etcd install and configure certificates
1 Environmental ready
系统 centos7.3
master 192.168.18.103 (所按照的软件:etcd,kube-apiserver,kube-controller-manager,kube-scheduler)
node1 192.168.18.104 (所按照的软件:etcd,docker,kublet,kube-proxy,flannel)
node2 192.168.18.105 (所按照的软件:etcd,docker,kublet,kube-proxy,flannel)
注意,这里我为了节省机器etcd这个我安装到了node节点上面,正常的话应该找单独的机器安装,三台)
Installation Docker 2 (node installed on two nodes)
yum install -y yum-utils device-mapper-persistent-data lvm2
yum-config-manager \
--add-repo \
https://download.docker.com/linux/centos/docker-ce.repo
yum install docker-ce -y
# 设置docker加速器
curl -sSL https://get.daocloud.io/daotools/set_mirror.sh | sh -s
http://bc437cce.m.daocloud.io
systemctl start docker
systemctl enable docker
docker version #查看版本
3 self-signed certificate etcd
1 Install the certificate generation tool cfssl
curl -L https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 -o /usr/local/bin/cfssl
curl -L https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 -o /usr/local/bin/cfssljson
curl -L https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64 -o /usr/local/bin/cfssl-certinfo
chmod +x /usr/local/bin/cfssl /usr/local/bin/cfssljson /usr/local/bin/cfssl-certinfo
2 certificate generation process, pay attention to that directory run the following command will generate a certificate under that directory, I am here to put / home / yx / ssl below
cat > ca-config.json <<EOF
{
"signing": {
"default": {
"expiry": "87600h"
},
"profiles": {
"www": {
"expiry": "87600h",
"usages": [
"signing",
"key encipherment",
"server auth",
"client auth"
]
}
}
}
}
EOF
cat > ca-csr.json <<EOF
{
"CN": "etcd CA",
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"L": "Beijing",
"ST": "Beijing"
}
]
}
EOF
cfssl gencert -initca ca-csr.json | cfssljson -bare ca -
#-----------------------
cat > server-csr.json <<EOF
{
"CN": "etcd",
"hosts": [
"192.168.18.103",
"192.168.18.104",
"192.168.18.105"
],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"L": "BeiJing",
"ST": "BeiJing"
}
]
}
EOF
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=www server-csr.json | cfssljson -bare server
4 Install etcd (three need to be installed)
1 Create a directory,
mkdir -p /home/yx/{ssl,bin,cfg}
2 Download etcd, decompression, and moved to the specified directory
下载地址 https://github.com/etcd-io/etcd/releases,我这里安装的是etcd-v3.2.12-linux-amd64.tar.gz
移动
mv etcd-v3.2.12-linux-amd64/etcd /home/yx/etcd/bin/
mv etcd-v3.2.12-linux-amd64/etcdctl /home/yx/etcd/bin/
3 Profiles
# 里面的ip根据自己的情况来定
[root@tidb-tidb-03 ~]# cat /home/yx/etcd/cfg/etcd
#[Member]
ETCD_NAME="etcd01"
ETCD_DATA_DIR="/home/yx/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="https://192.168.18.103:2380"
ETCD_LISTEN_CLIENT_URLS="https://192.168.18.103:2379"
#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.18.103:2380"
ETCD_ADVERTISE_CLIENT_URLS="https://192.168.18.103:2379"
ETCD_INITIAL_CLUSTER="etcd01=https://192.168.18.103:2380,etcd02=https://192.168.18.104:2380,etcd03=https://192.168.18.105:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new
4 copies of certificates
#把生成的证书拷贝到etcd的ssl目录下面
cp -r /home/yx/ssl/*.pem /home/yx/etcd/ssl
5 Configure etcd startup script
cat /usr/lib/systemd/system/etcd.service
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target
[Service]
Type=notify
EnvironmentFile=/home/yx/etcd/cfg/etcd #这个是你配置文件的路径
ExecStart=/home/yx/etcd/bin/etcd \
--name=${ETCD_NAME} \
--data-dir=${ETCD_DATA_DIR} \
--listen-peer-urls=${ETCD_LISTEN_PEER_URLS} \
--listen-client-urls=${ETCD_LISTEN_CLIENT_URLS},http://127.0.0.1:2379 \
--advertise-client-urls=${ETCD_ADVERTISE_CLIENT_URLS} \
--initial-advertise-peer-urls=${ETCD_INITIAL_ADVERTISE_PEER_URLS} \
--initial-cluster=${ETCD_INITIAL_CLUSTER} \
--initial-cluster-token=${ETCD_INITIAL_CLUSTER} \
--initial-cluster-state=new \
--cert-file=/home/yx/etcd/ssl/server.pem \
--key-file=/home/yx/etcd/ssl/server-key.pem \
--peer-cert-file=/home/yx/etcd/ssl/server.pem \
--peer-key-file=/home/yx/etcd/ssl/server-key.pem \
--trusted-ca-file=/home/yx/etcd/ssl/ca.pem \
--peer-trusted-ca-file=/home/yx/etcd/ssl/ca.pem
Restart=on-failure
LimitNOFILE=65536
6 Start and copy startup scripts and configuration files to rest on top of the two machines
systemctl start etcd
systemctl enable etcd
ps -ef | grep etcd
注意:如果只是其中一台启动,日志里面可能会提示信息,需要三台全部启动完毕即可。
# 如果启动报错,可以查看/var/log/message/ 或者journalctl -u etcd 查看错误信息
scp -r /usr/lib/systemd/system/etcd.service [email protected]:/usr/lib/systemd/system/
scp -r /home/yx/etcd/ [email protected]:/home/yx/ #这个里面包含了配置文件,证书等
7 Check etcd cluster status:
检测之前要进去到证书的目录下面,cd /home/yx/etcd/ssl
/home/yx/etcd/bin/etcdctl --ca-file=ca.pem --cert-file=server.pem --key-file=server-key-endpoints="https://192.168.18.103:2379,https://192.168.18.104:2379,https://192.168.18.1
member 257fc3f3cf354f5 is healthy: got healthy result from https://192.168.18.103:2379
member 49ea5453c8e9f751 is healthy: got healthy result from https://192.168.18.105:2379
member 59f641f0f8e3fd7a is healthy: got healthy result from https://192.168.18.104:2379
cluster is healthy
如果输出上面信息,就说明集群部署成功
Two clustered network deployment flannel
1 above the master operation, a write to the assigned subnet ETCD, for use flanneld
[yx@tidb-tidb-03 ssl]$ /home/yx/etcd/bin/etcdctl --ca-file=ca.pem --cert-file=server.pem --key-file=server-key.pem --endpoints="https://192.168.18.103:2379,https://192.168.18.104:2379,https://192.168.18.105:2379" set /coreos.com/network/config '{ "Network": "172.17.0.0/16", "Backend": {"Type": "vxlan"}}'
#产生下面信息为正常
{ "Network": "172.17.0.0/16", "Backend": {"Type": "vxlan"}}
2 Create a directory (operating in all node node above)
mkdir -p /home/yx/flannel/{cfg,ssl,bin}
3 Download and unzip the move
wget https://github.com/coreos/flannel/releases/download/v0.10.0/flannel-v0.10.0-linux-amd64.tar.gz
tar zxvf flannel-v0.9.1-linux-amd64.tar.gz
mv flanneld mk-docker-opts.sh /home/yx/flannel/bin
4 Configuration flannel profile
cat /home/yx/flannel/cfg/flanneld
FLANNEL_OPTIONS="--etcd-endpoints=https://192.168.18.103:2379,https://192.168.18.104:2379,https://192.168.18.105:2379 -etcd-cafile=/home/yx/etcd/ssl/ca.pem -etcd-certfile=/home/yx/etcd/ssl/server.pem -etcd-keyfile=/home/yx/etcd/ssl/server-key.pem"
5 systemd configuration management Flannel user starts flannel
# 注意里面的目录一定要写正确,不然会各种报错
[yx@tidb-tikv-01 ~]$ cat /usr/lib/systemd/system/flanneld.service
[Unit]
Description=Flanneld overlay address etcd agent
After=network-online.target network.target
Before=docker.service
[Service]
Type=notify
EnvironmentFile=/home/yx/flannel/cfg/flanneld
ExecStart=/home/yx/flannel/bin/flanneld --ip-masq $FLANNEL_OPTIONS
ExecStartPost=/home/yx/flannel/bin/mk-docker-opts.sh -k DOCKER_NETWORK_OPTIONS -d /run/flannel/subnet.env
Restart=on-failure
[Install]
WantedBy=multi-user.target
6 Docker start specified subnet configuration:, change it before you back up files
[yx@tidb-tikv-01 ~]$ cat /usr/lib/systemd/system/docker.service
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network-online.target firewalld.service
Wants=network-online.target
[Service]
Type=notify
EnvironmentFile=/run/flannel/subnet.env
ExecStart=/usr/bin/dockerd $DOCKER_NETWORK_OPTIONS
ExecReload=/bin/kill -s HUP $MAINPID
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
TimeoutStartSec=0
Delegate=yes
KillMode=process
Restart=on-failure
StartLimitBurst=3
StartLimitInterval=60s
[Install]
WantedBy=multi-user.target
7 Start flannel and docker
systemctl daemon-reload
systemctl start flanneld
systemctl enable flanneld
systemctl restart docker
8 Verify
[yx@tidb-tikv-01 ~]$ ps -ef |grep docker
root 977 1 0 17:19 ? 00:00:02 /usr/bin/dockerd --bip=172.17.62.1/24 --ip-masq=false --mtu=1450
root 984 977 0 17:19 ? 00:00:03 containerd --config /var/run/docker/containerd/containerd.toml --log-level info
yx 11000 30416 0 17:54 pts/0 00:00:00 grep --color=auto docker
再用ip add验证一下,确保docker0与flannel.1在同一网段。
As shown below
Then the current node to another node ping docker0 of ip, is through
9 no problem, then copy the files and scripts to another node node above
scp -r flannel/ [email protected]:/home/yx/
scp -r /usr/lib/systemd/system/flanneld.service [email protected]:/usr/lib/systemd/system/
scp -r /usr/lib/systemd/system/docker.service [email protected]:/usr/lib/systemd/system/