Docker containers across multiple networks host communications solutions

Network communication across a host program to achieve Docker there are many, such as before Bowen wrote through the realization of inter-host communication Docker container service deployment consul , but the consul and services can not be achieved across multiple network hosts to communicate, in this case below, MacVlan on the application of the concept of birth.

Macvlan works:

Macvlan network interface is supported by the Linux kernel. Linux build is required v3.9-3.19 and 4.0+;

By creating Macvlan subinterface physical NIC, allowing a plurality of separate physical NIC has a MAC address and IP address. Out of the virtual sub-interface directly exposed to the adjacent physical network. It seems from the outside, like the cable strands are separated, each received the same on different hosts;

After physical NIC receives a packet, the packet needs to be determined in which virtual network interface card according to the MAC address of the received packet.

When the container is directly connected to the physical network need to be used Macvlan. Macvlan itself does not create a network, the first host physical network card work in 'mixed mode', MAC address so that the physical network card will fail, flow all the physical Layer 2 network card can receive in nature. The next step is to create a virtual NIC on this physical network adapter, and specify the MAC address for a virtual network card to achieve a multi-purpose card, appears in the physical network, each virtual network adapter is a separate interface.

Use Macvlan there are several points to note:

The container is connected directly to the physical network, the network responsible for the allocation of the physical IP address, may be the result of physical network IP addresses is depleted Another consequence is that the network performance problem, the access to the host physical network increases, the proportion of fast broadcast packet liter high network performance problems caused by the fall;

Zhang online on a host needs to work in 'chaos mode';

As mentioned earlier, the physical NIC in the chaos work mode, the MAC address will fail, therefore, this mode of operation of the container and can not communicate with the external network, but does not affect the host and the external network communication;

In the long term bridge network and overlay network is a better choice, because that is a virtual network should be isolated from the physical network instead of sharing.

Work diagram is as follows:

Docker containers across multiple networks host communications solutions

Second, the configuration examples

Example 1 (host communication across the vessel to achieve a single network-based macvlan):

Achieve results:

Two centos 7.5, respectively running docker services;

Two docker server creates a MacVlan the same network, the container on the server can docker cross-host communication.

Start configuration:
(1) the first server docker configured as follows:

[root@docker ~]# ip link set ens33 promisc on    #开启ens33网卡的混杂模式。
#也就是开启网卡的多个虚拟interface（接口）  
[root@docker ~]# ip link show ens33    #确定查看的信息包含以下标红的字样
2: ens33: <BROADCAST,MULTICAST,'PROMISC',UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT qlen 1000
    link/ether 00:0c:29:2f:89:cb brd ff:ff:ff:ff:ff:ff
[root@docker ~]# docker network create -d macvlan --subnet 172.22.16.0/24 --gateway 172.22.16.1 -o parent=ens33 mac_net1
#创建macvlan网络，指定网段、网关等信息，“-o”指定绑定在哪张网卡之上
[root@docker ~]# docker run -tid --name box1 --ip 172.22.16.10 --network mac_net1 busybox
#基于新创建的macvlan网络运行一个容器，并指定其IP。

IP address confirmation operation of the container:
Docker containers across multiple networks host communications solutions
(1) arranged below a second docker server (the configuration server docker first substantially similar):

[root@docker02 ~]# ip link set ens33 promisc on      #开启ens33网卡的混杂模式。
[root@docker02 ~]#  ip link show ens33      #确定查看的信息包含以下标红的字样
2: ens33: <BROADCAST,MULTICAST,'PROMISC',UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT qlen 1000
    link/ether 00:0c:29:c6:57:bc brd ff:ff:ff:ff:ff:ff
[root@docker02 ~]# docker network create -d macvlan --subnet 172.22.16.0/24 --gateway=172.22.16.1 -o parent=ens33 mac_net1
#创建一个与第一台docker服务器的网段、网关相同的macvlan。并绑定到物理网卡上。
#为了可以直观的看出其他docker服务器上的macvlan和第这台是在同一个网段的。
#所以，建议设置的网络名称一样。
[root@docker02 ~]# docker run -tid --name box2 --ip 172.22.16.11 --network mac_net1 busybox
#运行一个容器，并指定是基于macvlan网络的
#注意，其IP地址不要与其他docker服务器上的容器IP地址冲突

Confirm container running IP address:
Docker containers across multiple networks host communications solutions

Box2 container using a second server on the docker box1 container on the first server docker ping test:

Docker containers across multiple networks host communications solutions

OK, container cross-host communication is realized by macvlan. The use of promiscuous mode will cause the MAC address of the physical network card failure, the container, and this mode can not communicate with the external network, how the network communication with the outside, after Bowen will be written.

Example 2 (host-based network across multiple networks macvlan of the solution)

The effect achieved is as follows:

Two centos 7.5, respectively running docker services;

Each host MacVlan network for creating two containers (172.10.16.0/24 and 172.20.16.0/24);

And operating the container bbox1 docker bbox2 on the first server, a second server running docker container bbox3 and bbox4.

Ultimately communicate with each other across the same network segment of the host vessel.

Start configuration:
(1) the first server docker configured as follows:

[root@docker ~]# ip link set ens33 promisc on    #开启ens33网卡的混杂模式。
#也就是开启网卡的多个虚拟interface（接口）  
[root@docker ~]# ip link show ens33    #确定查看的信息包含以下标红的字样
2: ens33: <BROADCAST,MULTICAST,'PROMISC',UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT qlen 1000
    link/ether 00:0c:29:2f:89:cb brd ff:ff:ff:ff:ff:ff
[root@docker ~]# modinfo 8021q    #查看是否加载8021q模块，如果有信息返回，则表示该模块已经加载

Information modinfo 8021q command returns the following:
Docker containers across multiple networks host communications solutions

[root@docker ~]# modprobe 8021q     #若没有加载8021q模块，则执行此命令
[root@docker ~]# cd /etc/sysconfig/network-scripts/
[root@docker network-scripts]# vim ifcfg-ens33     #更改物理网卡配置
            ...............#省略部分内容
BOOTPROTO=manual        #将此配置项该为“manual”，也是手动的意思
            ...............#省略部分内容
#更改完成后，保存退出即可
[root@docker network-scripts]# cp -p ifcfg-ens33 ifcfg-ens33.10   #将网卡配置文件复制一份
# “-p”表示保留文件原本的属性
[root@docker network-scripts]# vim ifcfg-ens33.10     #更改复制出来的配置文件如下

BOOTPROTO=none
NAME=ens33.10       #注意更改名称
DEVICE=ens33.10     #注意更改名称
ONBOOT=yes
IPADDR=192.168.10.10    #给虚拟网卡设置一个IP
PREFIX=24
GATEWAY=192.168.10.2
VLAN=yes
#更改完成后，保存退出即可，注意，以上的IP与容器将要使用的IP并不是同一网段
[root@docker network-scripts]# cp ifcfg-ens33.10 ifcfg-ens33.20
[root@docker network-scripts]# vim ifcfg-ens33.20    #编辑如下
BOOTPROTO=none
NAME=ens33.20       #注意更改名称
DEVICE=ens33.20       #注意更改名称
ONBOOT=yes
IPADDR=192.168.30.10    #注意，此处的IP与ens33.10并不在同一网段
PREFIX=24
GATEWAY=192.168.30.2
VLAN=yes
#更改完成后，保存退出即可 
[root@docker network-scripts]# ifdown ens33;ifup ens33   #重启网卡，使更改生效
[root@docker network-scripts]# ifup ens33.10    #开启ens33.10
[root@docker network-scripts]# ifup ens33.20    #开启ens33.20
[root@docker ~]# docker network create -d macvlan --subnet 172.10.16.0/24 --gateway 172.10.16.1 -o parent=ens33.10 mac_net10
#创建一个macvlan网络，给其定义一个网段、网关及绑定到ens33.10
[root@docker ~]# docker network create -d macvlan --subnet 172.20.16.0/24 --gateway 172.20.16.1 -o parent=ens33.20 mac_net20
#创建一个macvlan网络，给其定义一个网段、网关及绑定到ens33.20
#接下来分别基于刚刚创建的macvlan网络运行一个容器
[root@docker ~]# docker run -itd --name bbox1 --network mac_net10 --ip 172.10.16.10 busybox
#基于网络mac_net10运行一个容器，并指定其IP
[root@docker ~]# docker run -itd --name bbox2 --network mac_net20 --ip 172.20.16.20 busybox
#基于网络mac_net10运行一个容器，并指定其IP

(2) a second server configured as follows docker (substantially similar to the first stage operation, but to pay attention not to conflict IP):

[root@docker02 ~]# ip link set ens33 promisc on    #开启ens33网卡的混杂模式。
#也就是开启网卡的多个虚拟interface（接口）  
[root@docker02 ~]# ip link show ens33    #确定查看的信息包含以下标红的字样
2: ens33: <BROADCAST,MULTICAST,'PROMISC',UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT qlen 1000
    link/ether 00:0c:29:2f:89:cb brd ff:ff:ff:ff:ff:ff
[root@docker02 ~]# modinfo 8021q    #查看是否加载8021q模块，如果有信息返回，则表示该模块已经加载

Information modinfo 8021q command returns the following:
Docker containers across multiple networks host communications solutions

[root@docker02 ~]# modprobe 8021q     #若没有加载8021q模块，则执行此命令
[root@docker02 ~]# cd /etc/sysconfig/network-scripts/
[root@docker02 network-scripts]# vim ifcfg-ens33     #更改物理网卡配置
            ...............#省略部分内容
BOOTPROTO=manual        #将此配置项该为“manual”，也是手动的意思
            ...............#省略部分内容
#更改完成后，保存退出即可
[root@docker02 network-scripts]# scp [email protected]:/etc/sysconfig/network-scripts/ifcfg-ens33.* .
#将第一台docker服务器上的虚拟网卡配置文件复制过来

[email protected] s password:         #输入第一台docker服务器的用户密码
ifcfg-ens33.10                 100%  117     0.1KB/s   00:00    
ifcfg-ens33.20                 100%  117     0.1KB/s   00:00    
[root@docker02 network-scripts]# vim ifcfg-ens33.10    #只是更改其IP即可

BOOTPROTO=none
NAME=ens33.10
DEVICE=ens33.10
ONBOOT=yes
IPADDR=192.168.10.11      #更改IP是为了不要和第一台docker服务器的虚接口IP冲突
PREFIX=24
GATEWAY=192.168.10.2
VLAN=yes
[root@docker02 network-scripts]# vim ifcfg-ens33.20       #同上

BOOTPROTO=none
NAME=ens33.20
DEVICE=ens33.20
ONBOOT=yes
IPADDR=192.168.30.11    #更改其IP地址
PREFIX=24
GATEWAY=192.168.30.2
VLAN=yes
[root@docker02 network-scripts]# ifdown ens33;ifup ens33   #重启网卡，使更改生效
[root@docker02 network-scripts]# ifup ens33.10   #启动ens33.10
[root@docker02 network-scripts]# ifup ens33.20   #启动ens33.20
#接下来创建macvlan网络，与第一台docker服务器创建的网络一样
[root@docker02 ~]# docker network create -d macvlan --subnet 172.10.16.0/24 --gateway 172.10.16.1 -o parent=ens33.10 mac_net10
[root@docker02 ~]# docker network create -d macvlan --subnet 172.20.16.0/24 --gateway 172.20.16.1 -o parent=ens33.20 mac_net20
#接下来基于刚刚创建的网络运行两个容器
#基于mac_net10运行容器bbox3
[root@docker02 ~]# docker run -tid --name bbox3 --network mac_net10 --ip 172.10.16.11 busybox
#基于mac_net20运行容器bbox4
[root@docker02 ~]# docker run -itd --name bbox4 --network mac_net20 --ip 172.20.16.21 busybox

Configuration At this point, the ping test can be carried out, if configured correctly, it should bbox3 and bbox1 interoperability (because it is based on mac_net1010 network); bbox4 should bbox2 exchange (empathy).

But bbox3 and bbox1 can not bbox4 and bbox2 interoperability (because it is not based on the same virtual local area network).

Bbox3 ping container vessel bbox1 test (Note: If using vmware virtual machine for testing, due to the characteristics of vmware, its network adapters need to be changed to "bridge mode" rather than NAT mode or can not communicate.) :
Docker containers across multiple networks host communications solutions

Container vessel bbox2 bbox4 ping test:
Docker containers across multiple networks host communications solutions

At this point, the host network across multiple networks has been achieved, likewise, each container can not communicate with the external network, as the solution to this problem, I will write later.

docker network is a complex concept, if so be patient, it is recommended to read docker official documents

-------- end of this article so far, thanks for reading --------