Introduction to overlay network
The overlay network is the application layer network, which is oriented to the application layer, and does not or rarely considers the network layer and the physical layer.
In detail, an overlay network refers to a network built on another network. The nodes in the network can be seen as connected through virtual or logical links. Although there are many physical links at the bottom layer, these virtual or logical links all correspond to paths one-to-one. For example: Many P2P networks are overlay networks because they run on the upper layer of the Internet. The overlay network allows routing information to destination hosts without IP address identification. For example, Freenet and DHT (distributed hash table) can route information to a node that stores specific files, and the IP address of this node is not known in advance.
The overlay network is considered to be a way to improve the routing of the Internet, allowing the Layer 2 network to pass through the Layer 3 network, which not only solves the shortcomings of Layer 2 but also solves the inflexibility of Layer 3!
How Flannel works
Flannel is essentially an "overlay network", that is, TCP data is packaged in another network packet for routing, forwarding and communication. It currently supports data forwarding methods such as UDP, VxLAN, AWS VPC, and GCE routing. .
The default data communication method between nodes is UDP forwarding.
working principle
After the data is sent from the source container, it is forwarded to the flannel0 virtual network card via the docker0 virtual network card of the host where it is located. This is a P2P virtual network card, and the flanneld service is monitored on the other end of the network card.
Flannel maintains a routing table between nodes through the Etcd service, which records in detail the subnet segments of each node.
The flanneld service of the source host encapsulates the original data content in UDP and delivers it to the flanneld service of the destination node according to its own routing table. After the data arrives, it is unpacked, and then directly enters the flannel0 virtual network card of the destination node, and then is forwarded to the destination host. docker0 virtual network card, finally, just like the local container communication, there is docker0 route to reach the target container.
Configuration file
/etc/sysconfig/flanneld
[root@k8s-master ~]# vi /etc/sysconfig/flanneld
Flanneld configuration options
etcd url location. Point this to the server where etcd runs
FLANNELETCDENDPOINTS="http://etcd:2379"
etcd config key. This is the configuration key that flannel queries
For address range assignment
FLANNELETCDPREFIX="/atomic.io/network"
Any additional options that you want to pass
FLANNEL_OPTIONS=""
Flannel uses Etcd for configuration to ensure the configuration consistency between multiple Flannel instances, so you need to configure the following on etcd: ('/atomic.io/network/config' This key is the same as the above /etc/sysconfig/flannel The configuration item FLANNELETCDPREFIX in is corresponding, if it is wrong, there will be an error when starting)
[root@k8s-master ~]# etcdctl mk /atomic.io/network/config '{ "Network": "10.0.0.0/16" }' { "Network": "10.0.0.0/16" }
UDP packet encapsulation
Let's look at the following figure, which is a ping command communication data packet captured on one of the communication nodes. It can be seen that the data content part of UDP is actually another ICMP (that is, ping command) data packet.
The original data is encapsulated by UDP on the Flannel service of the starting node. After being delivered to the destination node, it is restored to the original data packet by the Flannel service on the other end. The Docker services on both sides cannot feel the existence of this process.
docker IP allocation
After Flannel allocated the available IP address range for each node through Etcd, it secretly modified the startup parameters of Docker.
[root@k8s-node-1 ~]# ps aux | grep bip
root 3142 0.1 2.7 560620 27364 ? Ssl 19:50 0:11 /usr/bin/dockerd-current --add-runtime docker-runc=/usr/libexec/docker/docker-runc-current --default-runtime=docker-runc --exec-opt native.cgroupdriver=systemd --userland-proxy-path=/usr/libexec/docker/docker-proxy-current --seccomp-profile=/etc/docker/seccomp.json --insecure-registry registry:5000 --storage-driver overlay2 --bip=10.0.53.1/24 --ip-masq=true --mtu=1472
This is the running parameters of the Docker service process viewed on the node running the Flannel service.
Note the "--bip=10.0.53.1/24" parameter, which limits the IP range obtained by the node container.
This IP range is automatically assigned by Flannel, and Flannel ensures that they will not be duplicated through the records saved in the Etcd service.
The container IP is not fixed. Docker is still doing the IP allocation, and Flannel just allocates the subnet segment.
Data forwarding
The following is the routing table of the two node nodes of the k8s cluster:
[root@k8s-node-1 ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.1.1 0.0.0.0 UG 100 0 0 ens33
10.0.0.0 0.0.0.0 255.255.0.0 U 0 0 0 flannel0
10.0.53.0 0.0.0.0 255.255.255.0 U 0 0 0 docker0
192.168.1.0 0.0.0.0 255.255.255.0 U 100 0 0 ens33
[root@k8s-node-2 ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.1.1 0.0.0.0 UG 100 0 0 ens33
10.0.0.0 0.0.0.0 255.255.0.0 U 0 0 0 flannel0
10.0.80.0 0.0.0.0 255.255.255.0 U 0 0 0 docker0
192.168.1.0 0.0.0.0 255.255.255.0 U 100 0 0 ens33
For example: Now there is a data packet to be sent from the container with IP 10.0.53.2 to the container with IP 10.0.80.2. According to the routing table of the data sending node, it only matches the 10.0.0.0/16 record, so the data is delivered to flannel0 after coming out of docker0. Similarly, at the target node, since the delivery address is a container, the destination address must fall on the 10.0.80.0/24 record of docker0, and then delivered to the docker0 network card
Installation and configuration
Execute the following commands on master and node to install
[root@k8s-master ~]# yum install flannel
Configure Flannel
Edit /etc/sysconfig/flanneld on both master and node
[root@k8s-master ~]# vi /etc/sysconfig/flanneld
Flanneld configuration options
etcd url location. Point this to the server where etcd runs
FLANNELETCDENDPOINTS="http://etcd:2379"
etcd config key. This is the configuration key that flannel queries
For address range assignment
FLANNELETCDPREFIX="/atomic.io/network"
Any additional options that you want to pass
FLANNEL_OPTIONS=""
Configure the key of flannel in etcd
Flannel uses Etcd for configuration to ensure the configuration consistency between multiple Flannel instances, so you need to configure the following on etcd: ('/atomic.io/network/config' This key is the same as the above /etc/sysconfig/flannel The configuration item FLANNELETCDPREFIX in is corresponding, if it is wrong, there will be an error when starting)
[root@k8s-master ~]# etcdctl mk /atomic.io/network/config '{ "Network": "10.0.0.0/16" }'
{ "Network": "10.0.0.0/16" }
start up
After starting Flannel, you need to restart docker and kubernete in turn.
Execute in master:
systemctl enable flanneld.service
systemctl start flanneld.service
service docker restart
systemctl restart kube-apiserver.service
systemctl restart kube-controller-manager.service
systemctl restart kube-scheduler.service
Execute on node:
systemctl enable flanneld.service
systemctl start flanneld.service
service docker restart
systemctl restart kubelet.service
systemctl restart kube-proxy.service
Author: ywhu
link: https: //www.jianshu.com/p/165a256fb1da
Source: Jane book
Jane book copyright reserved by the authors, are reproduced in any form, please contact the author to obtain authorization and indicate the source.