docker falnnel installation and finishing

Flannel for Docker

Flannel , similar to Calico , VXLAN and Weave , provides a configurable virtual overlay network for Docker . Flannel runs an agent on each host, flanneld , which is responsible for allocating subnet leases in the preconfigured address space. Flannel uses etcd to store network configuration. Let's take a look at the diagram on the Flannel GitHub to describe the details of database transfers in overlapping networks .

Download and run Etcd

Since Flannel relies on Etcd , we need to configure Etcd before starting flanneld , we need two Linux hosts (bare metal or virtual machine), the host name and IP are: Node1:192.168.56.10 , Node2:192.168.56.20 , on the two nodes Download and run Etcd : Download etcd

$ curl -L https://github.com/coreos/etcd/releases/download/v2.3.6/etcd-v2.3.6-linux-amd64.tar.gz -o etcd-v2.3.6-linux-amd64.tar.gz

$ tar zxvf etcd-v2.3.6-linux-amd64.tar.gz

$ cd etcd-v2.3.6-linux-amd64/

NODE1

./etcd -name node1 -initial-advertise-peer-urls http://192.168.56.10:2380 \

-listen-peer-urls http://0.0.0.0:2380 \

-listen-client-urls http://0.0.0.0:2379,http://127.0.0.1:4001 \

-advertise-client-urls http://0.0.0.0:2379 \

-initial-cluster-token etcd-cluster \

-initial-cluster node1=http://192.168.56.10:2380,node2=http://192.168.56.20:2380 \

-initial-cluster-state new

NODE2

./etcd -name node2 -initial-advertise-peer-urls http://192.168.56.20:2380 \

-listen-peer-urls http://0.0.0.0:2380 \

-listen-client-urls http://0.0.0.0:2379,http://127.0.0.1:4001 \

-advertise-client-urls http://0.0.0.0:2379 \

-initial-cluster-token etcd-cluster \

-initial-cluster node1=http://192.168.56.10:2380,node2=http://192.168.56.20:2380 \

-initial-cluster-state new

Configure Etcd
Flannel to read the configuration from etcd , the default is to read the configuration from /coreos.com/network/config (can be overridden by --etcd-prefix ), we need to use the etcdctl tool to set the value in etcd, in the directory where etcd is downloaded , run the following command:

./etcdctl set /coreos.com/network/config '{ "Network": "10.0.0.0/8","SubnetLen": 20, "SubnetMin": "10.10.0.0","SubnetMax": "10.99.0.0","Backend": { "Type": "udp", "Port": 7890 } }'

我们可以在node2上查看上面保存的值

$ ./etcdctl get /coreos.com/network/config

{ "Network": "10.0.0.0/8","SubnetLen": 20, "SubnetMin": "10.10.0.0","SubnetMax": "10.99.0.0","Backend": { "Type": "udp", "Port": 7890 } }

构造运行Flannel

wget https://github.com/coreos/flannel/releases/download/v0.5.5/flannel-0.5.5-linux-amd64.tar.gz

tar xvf flannel-0.5.5-linux-amd64.tar.gz

运行Flannel
Etcd配置好后，我们在两台节点上运行 flanneld

$ ./flanneld &
使用ifconfig来确认flanned的网络配置成功，输出应该类似于：

flannel0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00

inet addr:10.14.128.0 P-t-P:10.14.128.0 Mask:255.0.0.0

UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1472 Metric:1

RX packets:0 errors:0 dropped:0 overruns:0 frame:0

TX packets:0 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:500

RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

Flannel 运行后，我们需要为docker0配置网络并使用Flannel网络重新启动docker进程

$ sudo service docker stop

$ source /run/flannel/subnet.env

$ sudo ifconfig docker0 ${FLANNEL_SUBNET}

$ sudo docker daemon --bip=${FLANNEL_SUBNET} --mtu=${FLANNEL_MTU} &

--bip=${FLANNEL_SUBNET} --mtu=${FLANNEL_MTU} &

参数加到docker启动OPTIONS中也可以

启动Docker

Flannel配置好后，我们就可以启动docker了，跟没有Flannel的时候一样。
Node1

$ docker run -itd --name=worker-1 ubuntu:14.04

$ docker run -itd --name=worker-2 ubuntu:14.04

Node2

$ docker run -itd –name=worker-3 ubuntu:14.04

我们使用sudo docker exec worker-N ifconfig来获取IP地址，在node1上，测试到work-3的连通性。
worker-1: 10.16.48.2
worker-2: 10.16.48.3
worker-3: 10.14.128.2

ubuntu@node1:~$ sudo docker exec worker-2 ping -c2 10.14.128.2

PING 10.14.128.2 (10.14.128.2) 56(84) bytes of data.

64 bytes from 10.14.128.2: icmp_seq=1 ttl=60 time=0.948 ms

64 bytes from 10.14.128.2: icmp_seq=2 ttl=60 time=1.07 ms

--- 10.14.128.2 ping statistics ---

2 packets transmitted, 2 received, 0% packet loss, time 1000ms

rtt min/avg/max/mdev = 0.948/1.012/1.076/0.064 ms

ubuntu@node1:~$ sudo docker exec worker-2 ping -c2 www.baidu.com

PING www.a.shifen.com (220.181.112.244) 56(84) bytes of data.

64 bytes from 220.181.112.244: icmp_seq=1 ttl=46 time=3.59 ms

64 bytes from 220.181.112.244: icmp_seq=2 ttl=46 time=3.46 ms

--- www.a.shifen.com ping statistics ---

2 packets transmitted, 2 received, 0% packet loss, time 1001ms

rtt min/avg/max/mdev = 3.463/3.528/3.593/0.065 ms

简单的性能测试

至此对于Docker的 Flannel配置完成，所有的容器也能够互相连通，下面我们做一个简单的测试

首先我们看一下在本地网络上的性能：

ubuntu@node1:~$ iperf -c 192.168.56.10

------------------------------------------------------------

Client connecting to 192.168.56.10, TCP port 5001

TCP window size: 2.50 MByte (default)

------------------------------------------------------------

[ 4] local 192.168.56.10 port 5001 connected with 192.168.56.10 port 45555

[ 3] local 192.168.56.10 port 45555 connected with 192.168.56.10 port 5001

[ ID] Interval Transfer Bandwidth

[ 3] 0.0-10.0 sec 17.7 GBytes 15.2 Gbits/sec

[ 4] 0.0-10.0 sec 17.7 GBytes 15.2 Gbits/sec

docker 在不同的主机上

root@6eafb8bb4e78:/# iperf -c 10.14.128.2

------------------------------------------------------------

Client connecting to 10.14.128.2, TCP port 5001

TCP window size: 85.0 KByte (default)

------------------------------------------------------------

[ 3] local 10.16.48.2 port 56516 connected with 10.14.128.2 port 5001

[ ID] Interval Transfer Bandwidth

[ 3] 0.0-10.0 sec 106 MBytes 89.1 Mbits/sec

docker在相同的主机上

root@6eafb8bb4e78:/# iperf -c 10.16.48.2

------------------------------------------------------------

Client connecting to 10.16.48.2, TCP port 5001

TCP window size: 2.50 MByte (default)

------------------------------------------------------------

[ 3] local 10.16.48.2 port 49316 connected with 10.16.48.2 port 5001

[ ID] Interval Transfer Bandwidth

[ 3] 0.0-10.0 sec 18.9 GBytes 16.3 Gbits/sec

Flannel支持两种后端：UDP后端和VxLAN后端，试一下VxLAN后端，速度会快很多并接近本地网络性能。

UDP 和 VxLAN 后端

Flannel有两种不同的后端，前面配置的是UDP后端，那是比较慢的方案，因为所有的包都是在用户空间中封装的。VxLAN后端使用Linux内核VxLAN支持，一些硬件特性来实现更快的网络。
非常容易切换到VxLAN后端，在配置Etcd时，在定义 backend时使用 vxlan。

./etcdctl set /coreos.com/network/config '{ "Network": "10.0.0.0/8","SubnetLen": 20, "SubnetMin": "10.10.0.0","SubnetMax": "10.99.0.0","Backend": { "Type": "vxlan", "Port": 7890 } }'

#启动etcd

./etcd -name infra0 -initial-advertise-peer-urls http://172.26.11.223:2380 \
-listen-peer-urls http://172.26.11.223:2380 \
-listen-client-urls http://172.26.11.223:2379,http://127.0.0.1:2379 \
-advertise-client-urls http://172.26.11.223:2379 \
-initial-cluster infra0=http://172.26.11.223:2380 



#启动etcd
/root/flannel-0.5.5/flanneld  -etcd-endpoints=http://172.26.11.224:2379

/root/flannel-0.5.5/flanneld  -etcd-endpoints=http://172.26.11.223:2379


#查看etcd
/root/etcd-v2.3.2-linux-amd64/etcdctl --peers 172.26.11.223:2379  get /coreos.com/network/config

#查看etcd
./etcdctl --peers 172.26.11.223:2379 ls /coreos.com/network/config
#设置etcd
/root/etcd-v2.3.2-linux-amd64/etcdctl --peers 172.26.11.223:2379  set /coreos.com/network/config '{ "Network": "10.0.0.0/8","SubnetLen": 20, "SubnetMin": "10.10.0.0","SubnetMax": "10.99.0.0","Backend": { "Type": "udp", "Port": 7890 } }'


#启动脚本

service docker stop                   #停止docker服务
/root/flannel-0.5.5/flanneld  -etcd-endpoints=http://172.26.11.224:2379               #启动flannel服务
/root/flannel-0.5.5/mk-docker-opts.sh -i                      #生成环境变量
source /run/flannel/subnet.env            #将环境变量生效
ifconfig docker0 ${FLANNEL_SUBNET}        #设置docker0的网卡ip
service docker start                 #启动docker服务


#修改配置文件
vi /usr/lib/systemd/system/docker.service

#docker重新加载配置文件
systemctl daemon-reload

#启动
docker run -itd --name=worker-1 reg.docker.tude.com/cmall/tomcat_jdk_base:latest

#查看worker-1容器ip
docker exec worker-1 ifconfig


#查看变量
echo "--bip=${FLANNEL_SUBNET} --mtu=${FLANNEL_MTU} "

docker daemon=true --bip=${FLANNEL_SUBNET} --mtu=${FLANNEL_MTU}

#ping
docker exec worker-1 ping -c2 10.10.16.3

docker exec worker-1 ping -c2 10.14.128.2

docker exec stoic_hugle ping -c2 10.10.16.2

docker falnnel installation and finishing

Guess you like