Docker across the host network --manual

Introduction 1. Macvlan

Before Macvlan happens, we can only add an Ethernet card multiple IP addresses, but can not add more MAC address, MAC address because it is by its globally unique identifier to an Ethernet card, even if you use to create ethx: this way y, you'll find all these "card," the MAC address and ethx are the same, in essence, they are still a piece of card, which will limit you to do a lot of operations of the second floor. With Macvlan technology, you can do it.

Macvlan allows you to configure multiple virtual network interfaces on a host's network interfaces, network interface which has its own independent MAC address, you can also configure the IP address for communication. Virtual machine, or the network and host vessel under Macvlan in the same segment, sharing the same broadcast domain. Macvlan and Bridge is quite similar, but because it eliminates the need for the existence Bridge, it is relatively simple to configure and debug, and the efficiency is relatively high. In addition, Macvlan itself perfect support VLAN.

The same data transmission through the inter-VLAN Layer visits, i.e. the MAC address implemented without using the route. Different VLAN unicast default user can not communicate directly, if you want to communicate, you need to do three routing equipment, Macvlan as well. Out of the virtual NIC by the virtual Macvlan technique, in logically and physically card is like. It is equivalent to a physical NIC switch, corresponding to the recorded virtual NIC MAC address and, when the physical NIC receives a packet, determines which of the packet belongs to a virtual network interface card according to the destination MAC address. This means that as long as the sub-interface sent from Macvlan packets (or packets destined Macvlan sub-interface), the physical card to receive only the data packet, the packet data is not processed, so this begs the question: Macvlan card can not be above the local IP and IP communication above the physical NIC! About the solution to this problem we discuss in the next section.
Docker across the host network --manual
In simple terms, Macvlan virtual network card device is a parasite on the physical network card device. Called when his contract contracting function, look to the physical device parasitic and contracting by the physical device. When receiving the packet, the physical device by the callback function rx_handler parasitic registration processing packets.

2. brief manual of procedures

macvlan As its name suggests, is a NIC virtualization technology, it is capable of a physical NIC plurality of virtual interfaces, each interface can be configured MAC address, each interface may be the same with its own the IP, each like the interface as a switch port, VLAN can be divided into it.

macvlan 的做法其实就是将这些虚拟出来的接口与 Docker 容器直连来达到通信的目的。一个 macvlan 网络对应一个接口,不同的 macvlan 网络分配不同的子网,因此,相同的 macvlan 之间可以互相通信,不同的 macvlan 网络之间在二层上不能通信,需要借助三层的路由器才能完成通信,如下,显示的就是两个不同的 macvlan 网络之间的通信流程。
Docker across the host network --manual
我们用一个 Linux 主机,通过配置其路由表和 iptables,将其配成一个路由器(当然是虚拟的),就可以完成不同 macvlan 网络之间的数据交换,当然用物理路由器也是没毛病的。

3.Macvlan 的特点:

1.可让使用者在同一张实体网卡上设定多个 MAC 地址。
2.承上,带有上述设定的 MAC 地址的网卡称为子接口(sub interface);而实体网卡则称为父接口(parent interface)。
3.parent interface 可以是一个物理接口(eth0),可以是一个 802.1q 的子接口(eth0.10),也可以是 bonding 接口。
4.可在 parent/sub interface 上设定的不只是 MAC 地址,IP 地址同样也可以被设定。
5.sub interface 无法直接与 parent interface 通讯 (带有 sub interface 的 VM 或容器无法与 host 直接通讯)。
承上,若 VM 或容器需要与 host 通讯,那就必须额外建立一个 sub 6.interface 给 host 用。
7.sub interface 通常以 mac0@eth0 的形式来命名以方便区別。
用张图来解释一下设定 Macvlan 后的样子:

Docker across the host network --manual

4.实验环境

docker01 docker02
192.168.1.11 192.168.1.13

关闭防火墙和禁用selinux,更改主机名

[root@localhost ~]# hostnamectl set-hostname docker01
[root@localhost ~]# su -
上一次登录:二 12月 17 08:20:36 CST 2019从 192.168.1.1pts/0 上
[root@docker01 ~]# systemctl  stop firealld
Failed to stop firealld.service: Unit firealld.service not loaded.
[root@docker01 ~]# setenforce 0
setenforce: SELinux is disabled
[root@docker01 ~]# systemctl daemon-reload 
[root@docker01 ~]# systemctl restart docker

4.1 macvlan的单网络通信

Docker across the host network --manual
1) 打开网卡的混杂模式
//需要在docker01和docker02_上都进行操作。

[root@docker01 ~]# ip link show ens33
//查看网卡模式

Docker across the host network --manual

[root@docker01 ~]# ip link set ens33 promisc on
//创建网卡模式为混杂模式
[root@docker01 ~]# ip link show ens33
//查看网卡模式

Docker across the host network --manual
2)在docker01.上创建macvlan网络

[root@docker01 ~]# docker network create -d macvlan --subnet 172.22.16.0/24 --gateway 172.22.16.1  -o parent=ens33 mac_net1
// 创建一个macvlan模式的网络
-o parent=绑定在哪张网卡之上
[root@docker01 ~]# docker network ls
//查看网卡信息

Docker across the host network --manual

3)基于创建的macvlan网络运行一个容器

[root@docker01 ~]# docker run -itd --name bbox1 --ip 172.22.16.10 --network mac_net1 busybox

4)在docker02.上创建macvlan网络(要和docker01的macvlan一模一样)

[root@docker02 ~]# docker network create -d macvlan  --subnet 172.22.16.0/24 --gateway 172.22.16.1  -o parent=ens33 mac_net1

[root@docker02 ~]# docker network ls

Docker across the host network --manual

5)在docker02. 上,基于创建的macvlan网络运行一个容器,验证与docker01.上容器的通信。

[root@docker02 ~]# docker run -itd --name bbox2 --network mac_net1 --ip 172.22.16.20 busybox
//基于busybox创建一个容器
[root@docker02 ~]# docker exec -it bbox2 /bin/sh
//进入bbox2容器
/ # ping 172.22.16.10
//ping一下docker01的主机

Docker across the host network --manual

4.2macvlan的多网络通信

Docker across the host network --manual

1) docker01和docker02验证内核模块8021q封装
macvlan需要解决的问题:基于真实的ens33网卡,生产新的虚拟网卡。

[root@docker01 ~]# modinfo 8021q
//验证内核模块8021q封装

Docker across the host network --manual

[root@docker01 ~]# modprobe 8021q
//如果内核模块没有开启,运行上边的命令导入一下

2)docker01基于ens33创建虚拟网卡
修改ens33网卡配置文件

[root@docker01 ~]# cd /etc/sysconfig/network-scripts/
[root@docker01 network-scripts]# vim ifcfg-ens33  

Docker across the host network --manual
手动添加虚拟网卡配置文件

[root@docker01 ~]# cd /etc/sysconfig/network-scripts/
[root@docker01 network-scripts]# cp -p ifcfg-ens33  ifcfg-ens33.10
//-p保留源文件或目录的属性
[root@docker01 network-scripts]# vim ifcfg-ens33.10
//修改ens33.10网卡配置文件
BOOTPROTO=none
NAME=ens33.10
DEVICE=ens33.10
ONBOOT=yes
IPADDR=192.168.10.10
PREFIX=24
GATEWAY=192.168.10.2
VLAN=yes

这里注意,IP要和ens33网段做一个区分, 保证网关和网段IP的一致性,设备名称和配置文件的-致性,并且打开VLAN支持模式。

创建第二个虚拟网卡配置文件

[root@docker01 network-scripts]# cp -p ifcfg-ens33.10 ifcfg-ens33.20
[root@docker01 network-scripts]# vim ifcfg-ens33.20
//修改ens33.20网卡配置文件
BOOTPROTO=none
NAME=ens33.20
DEVICE=ens33.20
ONBOOT=yes
IPADDR=192.168.20.20
PREFIX=24
GATEWAY=192.168.20.2
VLAN=yes

docker01上的操作,启用创建的虚拟网卡:

[root@docker01 network-scripts]# ifup ifcfg-ens33.10 
[root@docker01 network-scripts]# ifup ifcfg-ens33.20
[root@docker01 network-scripts]# ifconfig
//查看IP

Docker across the host network --manual

3)docker02基于ens33创建虚拟网卡
修改ens33网卡配置文件

[root@docker02 ~]# cd /etc/sysconfig/network-scripts/
[root@docker02 network-scripts]# vim ifcfg-ens33  

Docker across the host network --manual
手动添加虚拟网卡配置文件

[root@docker02 ~]# cd /etc/sysconfig/network-scripts/
[root@docker02 network-scripts]# cp -p ifcfg-ens33  ifcfg-ens33.10
//-p保留源文件或目录的属性
[root@docker02 network-scripts]# vim ifcfg-ens33.10
//修改ens33.10网卡配置文件
BOOTPROTO=none
NAME=ens33.10
DEVICE=ens33.10
ONBOOT=yes
IPADDR=192.168.10.11
PREFIX=24
GATEWAY=192.168.10.2
VLAN=yes

这里注意,IP要和ens33网段做一个区分, 保证网关和网段IP的一致性,设备名称和配置文件的-致性,并且打开VLAN支持模式。

创建第二个虚拟网卡配置文件

[root@docker02 network-scripts]# cp -p ifcfg-ens33.10 ifcfg-ens33.20
[root@docker02 network-scripts]# vim ifcfg-ens33.20
//修改ens33.20网卡配置文件
BOOTPROTO=none
NAME=ens33.20
DEVICE=ens33.20
ONBOOT=yes
IPADDR=192.168.20.21
PREFIX=24
GATEWAY=192.168.20.2
VLAN=yes

docker02上的操作,启用创建的虚拟网卡:

[root@docker02 network-scripts]# systemctl restart network
[root@docker02 network-scripts]# ifup ifcfg-ens33.10 
[root@docker02 network-scripts]# ifup ifcfg-ens33.20
[root@docker02 network-scripts]# ifconfig
//查看IP

Docker across the host network --manual
4)docekr01和docker02基于虚拟网卡,创建macvlan网络

[root@docker02 network-scripts]# docker network create -d macvlan --subnet 172.16.10.0/24 --gateway 172.16.10.1  -o parent=ens33.10 mac_net10
//创建一个新的网卡基于ens33.10
[root@docker02 network-scripts]# docker network create -d macvlan --subnet 172.16.20.0/24 --gateway 172.16.20.1  -o parent=ens33.20 mac_net20
//创建一个新的网卡基于ens33.20

5)Docker01部署一个私有仓库
Docker01

   72  docker pull registry
//下载registry镜像
   73  docker run -itd --name registry -p 5000:5000  --restart=always registry:latest 
//基于registry镜像,启动一台容器
   76  docker tag busybox:latest  192.168.1.11:5000/busybox:v1 
//把容器重命名一个标签
   77  docker ps

Docker across the host network --manual

   78  vim /usr/lib/systemd/system/docker.service #13行修改
ExecStart=/usr/bin/dockerd --insecure-registry 192.168.1.11:5000  

   80  systemctl daemon-reload 
   81  systemctl restart docker.service 
//重启docker
  100  docker push 192.168.1.11:5000/busybox:v1
//上传容器到私有仓库 
  101  docker images

Docker02

78  vim /usr/lib/systemd/system/docker.service #13行修改
ExecStart=/usr/bin/dockerd --insecure-registry 192.168.1.11:5000  

   80  systemctl daemon-reload 
   81  systemctl restart docker.service 
     //重启docker
   99  docker pull 192.168.1.11/busybox:v1 
     //下载刚刚上传的镜像

6)docker01和docker02基于busybox:v1镜像和网卡mac_net10,mac_net20,创建容器。
Docker01

[root@docker01 ~]# docker run -itd --name bbox10 --network mac_net10 --ip 172.16.10.10 192.168.1.11:5000/busybox:v1
[root@docker01 ~]# docker run -itd --name bbox20 --network mac_net20 --ip 172.16.20.20 192.168.1.11:5000/busybox:v1
**Docker02**
[root@docker02 ~]# docker run -itd --name bbox10 --network mac_net10 --ip 172.16.10.10 192.168.1.11:5000/busybox:v1
[root@docker02 ~]# docker run -itd --name bbox20 --network mac_net20 --ip 172.16.20.20 192.168.1.11:5000/busybox:v1

***这里只需注意,我们在这里的操作跟在docker01和上面的操作是一模一样的,操作顺序大致为:
验证8021q内核封装
基于ens33网卡创建新的虚拟网卡,ens33.10和ens33.20 (注意和docker01. 上的ens33.10和ens33.20必须是在同一-网段,且IP不能冲突)基于此网络运行容器。(注意和docker01 上的容器,都是基于刚刚创建的macvlan网络,但IP地址不能冲突)


7)验证
在docker01.上进入容器bbox10和docker02.上的bbox11进行通信。
在docker01.上进入容器bbox20和docker02.上的bbox21进行通信。
注意: VMware的网络必须设置为Bridge模式。

现在把docker01和docker02的网络模式设置为桥接模式

Docker across the host network --manual
测试一下相同网卡的主机是否能ping通

[root@docker01 ~]# docker exec -it bbox10  /bin/sh
/ # ping 172.16.20.20

Docker across the host network --manual

[root@docker02 ~]# docker exec -it bbox20  /bin/sh
/ # ping 172.16.20.20

Docker across the host network --manual

5.Macvlan 的局限性

Macvlan 是将 VM 或容器通过二层连接到物理网络的近乎理想的方案,但它也有一些局限性:

Switch 1.Linux host connection may limit the number of MAC addresses on the same physical port. Although you can allow network administrators to change these policies, but sometimes this method can not be implemented (for example, you go to the customer to do a quick demo of PoC).
2. Many NIC will have to limit the number of MAC addresses on the physical NIC. Beyond this limit will affect the performance of the system.
3.IEEE 802.11 do not like to have multiple MAC addresses on the same client, which means that your Macvlan sub-interfaces are unable to communicate in a wireless network card or the AP. You can break through complex approaches such restrictions, but there is a simpler way is to use Ipvlan, interested can access relevant information themselves.

6. Summary

macvlan virtualization technology is a network card, a card can be a virtual multiple network cards.

Macvlan particular communication mode, the common mode bridge.

In the Docker, macvlan only support bridge mode.

Macvlan may communicate the same, different macvlan floor can not communicate, the communication can be accomplished via three routes.

think for a while:

similarities and differences macvlan bridge and the bridge
there is a similar technique, multiple virtual NICs are sharing the same MAC address, but has a separate IP address, what is this technology?

Guess you like

Origin blog.51cto.com/14320361/2459313