Use weave to implement cross-host container connections
Weave is a solution developed by weaveworks to solve Docker cross-host network. It can create a virtual network to connect Docker containers on multiple hosts, so that the containers are like being connected to the same network switch. Those who use the network Applications do not have to configure information such as port mappings and links.
External devices can access services provided by application containers on the Weave network, and existing internal systems can also be exposed to application containers. Weave can penetrate firewalls and run on partially connected networks. In addition, Weave's communication supports encryption, so users can connect to the host from an untrusted network.
Weave will create a network bridge on the host. Each container is connected to the bridge through a veth pair. At the same time, there is a Weave router container connected to the bridge. The router will capture the network through the interface connected to the bridge. Get network packets (the interface works in Promiscuous mode).
A W (Weave router) is deployed on each host where Docker is deployed (it may be a physical machine or a virtual machine), which itself can also be deployed in the form of a container. When Weave runs, you can assign an IP and corresponding mask to each veth container. The bridge end of veth is the Weave router container, and the IP and mask are assigned during Weave launch.
The Weave network is composed of peer endpoints (peers) composed of these weave routers. Each peer has its own name, which includes a readable name for status and log output, and a unique identifier. Characters are used to distinguish each other during operation. Even if Docker is restarted, the host name will remain unchanged. These names default to the mac address.
Each host where a Weave router is deployed needs to open the firewall settings of TCP and UDP port 6783 to ensure that the control plane traffic and data plane traffic between the Weave routers can pass. The control plane consists of TCP connections established between weave routers, through which handshakes and topological relationship information are exchanged and communicated. This communication can be configured as encrypted communication. The data plane consists of UDP connections established between Weave routers, and most of these connections are encrypted. These connections are full-duplex and can traverse firewalls.
For each container in the weave network, weave will create a network bridge and create a veth pair between the bridge and each container. One end is added to the container's network namespace as a container network card, and an IP is configured for the container network card. With the corresponding mask, one end is connected to the bridge, and the traffic is finally forwarded to the opposite host through the weave router on the host.
The basic process is as follows:
1. Container traffic reaches the weave router bridge on the host through the veth pair.
2. Weave router uses pcap in mixed mode to intercept network data packets on the bridge, and excludes data traffic forwarded directly by the kernel through the bridge, such as data within this subnet, between local containers, and between the host and local containers. flow between. The captured packets are forwarded to the weave router of all other hosts through UDP.
3. On the receiving end, the weave router injects the packet into the interface on the bridge through pcap, and distributes the traffic to the container's network card through the veth pair on the bridge.
By default, weave is based on UDP to carry data packets between containers, and the network topology of the entire cluster can be completely customized. However, from a performance and usage perspective, there are still relatively large flaws:
The packaging and unpacking method of weave's custom container data packet is not versatile enough, the transmission efficiency is relatively low, and the performance loss is relatively large.
Cluster configuration is relatively load-intensive and requires manual construction of the network topology through the weave command line. In the case of large-scale clusters, it increases the burden on administrators.
Weave advantages and disadvantages:
- Advantages of Weave
- Supports encryption of communication between hosts.
- Supports containers to dynamically join or divest from the network.
- Supports multi-subnet communication across hosts.
- WeaveDisadvantages
You can only join the weave network through weave launch or weave connect.
Configuration operations
-
Build services
CPU name
system
Host IP
application services
master
CentOS7.4
192.168.56.128
Container: 192.168.10.10/24
slave1
CentOS7.4
192.168.56.130
Container: 192.168.10.11/24
1. Bridge mode requires the installation of bridge management tools
# yum install bridge-utils
2. Temporarily close the firewall and open port 6783 if necessary.
3. Host routing forwarding needs to be enabled.
#vim /etc/sysctl.conf
net.ipv4.ip_forward = 1
#sysctl -p
4. Download the binary package (required for all hosts)
# wget -O /usr/local/bin/weave https://raw.githubusercontent.com/zettio/weave/master/weave
Authorization:
# chmod a+x /usr/local/bin/weave
5. Initialize the Weave network
# weave launch
# docker ps -a
# brctl show
# ifconfig weave
# docker network ls
The data of weave is stored in a container named weavedb allocated on each machine. It is a data volume container and is only responsible for the persistence of data.
6. Connect to different hosts
The node host needs to be connected to the master host. You only need to follow the weave launch with the ip or hostname of the master host. The two machines will automatically establish a cluster and synchronize all required information.
(1) Execute the following command on slave1
# weave launch 192.168.56.128
# weave connect 192.168.56.128
# weave version
Check
# weave status
6. Weave has three ways to integrate with Docker so that running containers can run in the Weave network.
(1) Use the weave run command to run the container directly.
(2) Use the weave env command to modify the value of the DOKCER_HOST environment variable, so that docker client and weave interact, and weave and docker daemon interact, automatically configuring the network for the container, and making it transparent to users.
(3) Use weave plugin and use the --net=weave parameter when running the container.
1) Use the weave run command to run the container directly
After weave version 2.0, there is no "weave run" command.
2) First use docker run to start the container, and then use the weave attach command to bind the IP address to the container.
Use the following command on the master:
# docker run -dti --name weave01 --privileged centos:7.4.1708 /usr/sbin/init
bind ip address
#weave connect 192.168.56.130 //Configure the other party’s host IP
# weave attach 192.168.10.10/24 ba50b215c0ec
The error is reported as follows:
# docker exec -ti ba50b215c0ec /bin/bash
Use the following command on slave1:
#docker run -dti --name weave03 --privileged centos:7.4.1708 /usr/sbin/init
Bind IP address to container
# weave connect 192.168.56.128 //Configure the other host’s IP
# weave attach 192.168.10.11/24 6a26f76433b6
# docker exec -ti 6a26f76433b6 /bin/bash
Remark:
When docker run starts the container above, you can add the --net=none parameter. This means that after the container is started, the IP automatically assigned by docker0, the default virtual network card, is not used, but the IP bound by weave is used;
Of course, you can also choose not to add this parameter to start the container. In this way, after the container is started, there will be two network cards, that is, two IPs (as shown in the picture above):
One is the IP automatically assigned by docker0, which is suitable for communication between containers on the same host. That is, containers on the same host can communicate with each other using the IP assigned by docker0; the other is the IP bound to the weave bridge.
7. Test
Network test
Application testing
The host master logs into the 10 container to access the mysql service on the host slave1 container 11, as shown in the figure
8. Precautions
In the case of cross-host, containers in the same subnet can communicate with each other; but two containers in different subnets cannot be interconnected, even though the two containers cannot communicate on the same host!
The advantage of this is that different subnets are used for network isolation between containers.
Notes on using weave:
(1) Using Docker’s native network, you can access the host and the external network inside the container. That is to say, when starting the container, the virtual network card docker0 is used to allocate IP. In this case, after logging in to the container, you can ping the host IP and connect to the Internet!
However, on the host machine, the IP address of the docker0 bridge can be pinged, but the IP address of the weave bridge cannot be pinged. It can be used at this time
Use the "weave expose 192.168.0.1/24" command to add an IP to the weave bridge to achieve network connectivity between the container and the host. as follows
Configuration
Add the IP of the weave bridge on both master and slave1 machines
[root@master ~]# weave expose 192.168.10.1/2
[root@slave1 ~]# weave expose 192.168.10.1/24
Additional instructions:
(1) 192.168.10.1/24 represents the gateway of the weave container network segment
(2) #weave hide 192.168.10.1/24 means to overwrite or delete the currently specified configuration
Check
shortcoming:
(1) The IP address assigned to the container by weave will become invalid as the container is closed or restarted, and must be reconfigured if necessary.
(2)weave solves network problems, but in stand-alone mode