简介 这篇文章主要介绍了Docker&K8s---K8s跨宿主机pod通信之Flannel以及相关的经验技巧，文章约15127字，浏览量242，点赞数1，值得参考！
Docker&K8s—K8s跨宿主机pod通信之Flannel
Kubernetes设计了网络模型，但却将它的实现交给了网络插件，CNI网络插件最主要的功能就是实现POD资源能够跨宿主机进行通信。
常见的CNI网络插件：
- Flannel
- Calico
- Canal
- Contiv
- OpenContrail
- NSX-T
- Kube-router
Flannel
flannel的三种网络模型
-
host-gw模型：所有node ip必须在同一个物理网关设备下才能使用 它的原理就是：给宿主机添加一个静态路由，指明到达pod之前要经过的宿主机
-
Vxlan模型：
-
直接路由模型：当node不在同一个物理网关下，走vxaln模型,在同一个网关下，走host-gw模型
host-gw模型
12-11下载二进制包
[root@hdss68-21 ~]# cd /opt/src
[root@hdss68-21 src]# wget https://github.com/flannel-io/flannel/releases/download/v0.14.0/flannel-v0.14.0-linux-amd64.tar.gz
[root@hdss68-21 opt]# mkdir flannel-v0.11.0
[root@hdss68-21 src]# tar xf flannel-v0.11.0-linux-amd64.tar.gz -C /opt/flannel-v0.11.0
[root@hdss68-21 src]# cd /opt
[root@hdss68-21 opt]# ln -s /opt/flannel-v0.11.0 flannel
12-11拷贝证书，因为要通过etcd来通信
[root@hdss68-21 opt]# cd flannel
[root@hdss68-21 flannel]# ls
flanneld mk-docker-opts.sh README.md
[root@hdss68-21 flannel]# mkdir cert
[root@hdss68-21 flannel]# cd cert/
[root@hdss68-21 cert]# scp hdss68-200:/opt/certs/ca.pem .
root@hdss68-200's password:
ca.pem 100% 1342 108.3KB/s 00:00
[root@hdss68-21 cert]# scp hdss68-200:/opt/certs/client.pem .
root@hdss68-200's password:
client.pem 100% 1363 100.7KB/s 00:00
[root@hdss68-21 cert]# scp hdss68-200:/opt/certs/client-key.pem .
root@hdss68-200's password:
client-key.pem
配置文件&启动脚本
[root@hdss68-21 cert]# cd ..
[root@hdss68-21 flannel]# vim subnet.env
FLANNEL_NETWORK=172.68.0.0/16
FLANNEL_SUBNET=172.68.21.1/24
FLANNEL_MTU=1500
FLANNEL_IPMASQ=false
[root@hdss68-21 flannel]# vi flanneld.sh
#!/bin/sh
./flanneld \\
--public-ip=192.168.68.21 \\
--endpoints=https://192.168.68.12:2379,https://192.168.68.21:2379,https://192.168.68.22:2379 \\
--ca-file=./cert/ca.pem \\
--cert-file=./cert/client.pem \\
--key-file=./cert/client-key.pem \\
--iface=ens33 \\
--subnet-file=./subnet.env \\
--healthz-port=2401
[root@hdss68-21 flannel]# chmod u+x flanneld.sh
--endpoints=https://192.168.68.12:2379,https://192.168.68.21:2379,https://192.168.68.22:2379 \\
--etcd-prefix=/coreos.com/network
etcd增加host-gw模型
[root@hdss68-21 flannel]# cd /opt/etcd/
[root@hdss68-21 etcd]# ./etcdctl set /coreos.com/network/config '{"Network": "172.68.0.0/16", "Backend": {"Type": "host-gw"}}'
{"Network": "172.68.0.0/16", "Backend": {"Type": "host-gw"}}
[root@hdss68-21 etcd]# ./etcdctl get /coreos.com/network/config
{"Network": "172.68.0.0/16", "Backend": {"Type": "host-gw"}}
启动flannel
vi /etc/supervisord.conf
[program:flanneld-68-21]
command=/opt/flannel/flanneld.sh ; the program (relative uses PATH, can take args)
numprocs=1 ; number of processes copies to start (def 1)
directory=/opt/flannel ; directory to cwd to before exec (def no cwd)
autostart=true ; start at supervisord start (default: true)
autorestart=true ; retstart at unexpected quit (default: true)
startsecs=30 ; number of secs prog must stay running (def. 1)
startretries=3 ; max # of serial start failures (default 3)
exitcodes=0,2 ; 'expected' exit codes for process (default 0,2)
stopsignal=QUIT ; signal used to kill process (default TERM)
stopwaitsecs=10 ; max num secs to wait b4 SIGKILL (default 10)
user=root ; setuid to this UNIX account to run the program
redirect_stderr=true ; redirect proc stderr to stdout (default false)
stdout_logfile=/data/logs/flanneld/flanneld.stdout.log ; stderr log path, NONE for none; default AUTO
stdout_logfile_maxbytes=64MB ; max # logfile bytes b4 rotation (default 50MB)
stdout_logfile_backups=4 ; # of stdout logfile backups (default 10)
stdout_capture_maxbytes=1MB ; number of bytes in 'capturemode' (default 0)
stdout_events_enabled=false ; emit events on stdout writes (default false)
# 如果不能启动，最好重启一下然后关闭防火墙就会启动起来
supervisorctl update
supervisorctl status
另一种方式启动
# flanneld配置项
cat << EOF | tee /opt/kubernetes/cfg/flanneld
FLANNEL_OPTIONS="--etcd-endpoints=https://192.168.68.12:2379,https://192.168.68.21:2379,https://192.168.68.22:2379 \\
-etcd-cafile=/opt/flannel/cert/ca.pem \\
-etcd-certfile=/opt/flannel/cert/client.pem \\
-etcd-keyfile=/opt/flannel/cert/client-key.pem \\
-etcd-prefix=/coreos.com/network"
EOF
# flanneld启动服务
cat << EOF | tee /usr/lib/systemd/system/flanneld.service
[Unit]
Description=Flanneld overlay address etcd agent
After=network-online.target network.target
Before=docker.service
[Service]
Type=notify
EnvironmentFile=/opt/kubernetes/cfg/flanneld
ExecStart=/opt/kubernetes/bin/flanneld --ip-masq \\$FLANNEL_OPTIONS
ExecStartPost=/opt/kubernetes/bin/mk-docker-opts.sh -k DOCKER_NETWORK_OPTIONS -d /run/flannel/subnet.env
Restart=on-failure
[Install]
WantedBy=multi-user.target
EOF
# 启动
systemctl daemon-reload
systemctl start flanneld
systemctl enable flanneld
Flannel模型详解
host-gw模型
所有的主机必须要处在同一个网关下面，flannel就是在每个主机上加静态路由，让所有主机间可以互相通信。
VxLan模型
主机在两个不同的网关下，flannel会在宿主机上加一个虚拟ip（vip），然后通过解包在flannel网络隧道中传输到另一台主机的vip
直接路由模型
智能模型，会自动判断并选择是gw还是vxlan。