Networking in containers is an important part of building a docker cluster.
This article will introduce how to use flannel to implement multi-node intercommunication of containers.
The following figure is the implementation principle of flannel, taken from:
http://docker-k8s-lab.readthedocs.io/en/latest/docker/docker-flannel.html
There are two VMs in this article, one is the master node, which will install docker, etcd, and flannel, and the second one is the working node of docker, which will install docker and flannel. Specific steps are as follows:
one installation
1 node1
node1 will install docker, etcd and flannel
yum install docker, etcd,flannel -y
2 node2
node2 install docker, flannel
yum install docker,flannel -y
Second configuration to start etcd
1 Configuration
Configure etcd:
vim /etc/etcd/etcd.conf
ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:2379" #This parameter specifies the address and port for external services. 0.0.0.0 means that all interfaces can provide services
ETCD_ADVERTISE_CLIENT_URLS="http://192.168.235.128:2379" #This parameter indicates that other etcd nodes forward the link of this node
2 Start
start etcd
systemctl start etcd
3. Increase the kv value of the flannel network in etcd
To increase the kv value of the flannel network configuration through the etcdctl command, you need to write the configuration to the a.json file first:
etcdctl set /flannel/network/config < ./a.json
{"Network": "10.0.0.0/8",
"SubnetLen": 20,
"SubnetMin": "10.10.0.0",
"SubnetMax": "10.99.0.0",
"Backend": {"Type": "vxlan",
"VNI": 100,
"Port": 8472}
}
Four configuration flannel
1 Configure the flannel configuration file
Both nodes have the same configuration:
Prepare the log directory
mkdir /var/log/k8s/flannel
chmod 777 -R /var/log/k8s
Edit flannel configuration
vim /etc/sysconfig/flanneld
FLANNEL_ETCD_ENDPOINTS="http://192.168.235.128:2379" #etcd's ip address and port
FLANNEL_ETCD_PREFIX="/flannel/network" #Corresponding to the configuration in etcd just now
FLANNEL_OPTIONS="--logtostderr=false --log_dir=/var/log/k8s/flannel/ --iface=eno16777736" #log and on which port enable flannel
2 Start flannel
start flannel
systemctl start flannel
3 Check the port information
Check ifconfig at this point
docker0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1450
inet 172.18.0.1 netmask 255.255.240.0 broadcast 0.0.0.0
……
eno16777736: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.235.128 netmask 255.255.255.0 broadcast 192.168.235.255
……
flannel.100: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1450
inet 10.10.48.0 netmask 255.255.255.255 broadcast 0.0.0.0
……
You can see that a port of flannel.100 is generated.
4 Configure the address of docker0
Configure the docker0 port so that docker0 uses the address segment allocated by flannel:
source /run/flannel/subnet.env
echo ${FLANNEL_SUBNET}
ifconfig docker0 ${FLANNEL_SUBNET}
You can see that docker0 is in the network segment of flannel.100:
docker0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1450
inet 10.10.48.1 netmask 255.255.240.0 broadcast 0.0.0.0
eno16777736: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.235.128 netmask 255.255.255.0 broadcast 192.168.235.255
flannel.100: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1450
inet 10.10.48.0 netmask 255.255.255.255 broadcast 0.0.0.0
Five configuration ip forward
1 Configure iptables
Configure iptables settings to allow forward
iptables -P FORWARD ACCEPT
2 Configure the sysctl file
vim /etc/sysctl.conf
net.ipv4.ip_forward=1
sysctl –p
Six restart docker
systemctl restart docker
Seven checks
1 Start the container
On docker01:
docker run -d --name c01 httpd
On docker02:
docker run -d --name c02 httpd
2 Check network connectivity
On docker01:
docker exec -it c01 bash
ip a
root@d0a04613f4d9:/usr/local/apache2# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
11: eth0@if12: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
link/ether 02:42:0a:0a:30:02 brd ff:ff:ff:ff:ff:ff
inet 10.10.48.2/20 scope global eth0
You can see that the eth address 10.10.48.2 is the network segment of flannel.100
ping www.sina.com.cn
PING spool.grid.sinaedge.com (202.102.94.124) 56(84) bytes of data.
64 bytes from 202.102.94.124: icmp_seq=1 ttl=127 time=11.3 ms
64 bytes from 202.102.94.124: icmp_seq=2 ttl=127 time=11.9 ms
64 bytes from 202.102.94.124: icmp_seq=3 ttl=127 time=11.6 ms
On docker02:
root@60973d570c81:/usr/local/apache2# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
9: eth0@if10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
link/ether 02:42:0a:0b:b0:02 brd ff:ff:ff:ff:ff:ff
inet 10.11.176.2/20 scope global eth0
The same, 10.11.176.2 is the address assigned by flannel
ping each other:
ping 10.10.48.2
PING 10.10.48.2 (10.10.48.2) 56(84) bytes of data.
64 bytes from 10.10.48.2: icmp_seq=1 ttl=62 time=1.64 ms
64 bytes from 10.10.48.2: icmp_seq=2 ttl=62 time=1.32 ms
64 bytes from 10.10.48.2: icmp_seq=3 ttl=62 time=1.07 ms
c64 bytes from 10.10.48.2: icmp_seq=4 ttl=62 time=1.39 ms
can ping each other