1. Linux Bridge bridge management
Adding a bridge on multiple virtual machines, between a virtual machine can communicate with each other, but can also be a virtual machine on the external network.
kvm bridge management through command brctl
[root@localhost ~]# brctl ....... #桥 addbr <bridge> add bridge delbr <bridge> delete bridge #端口 addif <bridge> <device> add interface to bridge delif <bridge> <device> delete interface from bridge hairpin <bridge> <port> {on|off} turn hairpin on/off setageing <bridge> <time> set ageing time setbridgeprio <bridge> <prio> set bridge priority setfd <bridge> <time> set bridge forward delay sethello <bridge> <time> set hello time setmaxage <bridge> <time> set max message age setpathcost <bridge> <port> <cost> set path cost setportprio <bridge> <port> <prio> set port priority show [ <bridge> ] show a list of bridges showmacs <bridge> show a list of mac addrs showstp <bridge> show bridge stp info stp <bridge> {on|off} turn stp on/off [root@localhost ~]# brctl show bridge name bridge id STP enabled interfaces br0 8000.0050562266e7 no ens33 virbr0 8000.5254009483b2 yes virbr0-nic
2. VLAN
Represents LAN Local Area Network, a local area network, and Switch Hub is generally used to connect computers in a LAN. In general, two computers connected to the same Hub or Switch, they are in the same LAN.
A represents a LAN broadcast domain. The implication is: all members of the LAN will receive broadcast packets sent by any one member.
VLAN expressed Virtual LAN. switch having a VLAN function can be divided into a plurality of own port LAN. Broadcast packets can be sent from a computer to other computers in the same LAN received, but located in other LAN computers can not receive. Briefly, a VLAN switch is divided into a plurality of switches, limit the scope of the broadcast, in the isolated computer Layer to a different VLAN.
For example, there are two sets of machines, Group A and B, we want configured in Group A of the machine can access each other, Group B in the machine can also visit each other, but A and B machine can not visit each other. One method is to use two switches, A and B are connected to a switch. Another method is to use a switch with a VLAN function, the machine A and B are placed in a different VLAN.
VLAN isolation is the isolation story, A and B can not access each other refers to a Layer broadcast packet (such as ARP) can not cross the border of the VLAN. But the three-tier (for example, IP) can be achieved through a router so that A and B are interoperable.
Now almost all switches support VLAN. The switch port generally has two configuration modes: Access and Trunk. As shown below
Access opening
port only allows access through a VLAN. These ports are marked with VLAN tag, indicating that the port belongs to which VLAN. With different VLAN ID to distinguish VLAN, VLAN ID ranges are 1-4096. Access ports are directly connected to a computer network card, so the card out of the packet flows after Access ports are marked in the VLAN tag. Access port can only belong to one VLAN.
Trunk 口
Suppose there are two switches A and B. There are the A VLAN1 (red), VLAN2 (yellow), VLAN3 (blue); there VLAN1,2,3 on B, then how to make the same VLAN can communicate between the AB it?
The approach is to link A and B, and A and B port connected to allow data VLAN1,2,3 through all three VLAN. This port is a Trunk port. The VLAN1, data 2, 3 of the pack is always with their VLAN tag in the process of reaching the other side by the switch Trunk mouth.
multiple VLAN trunk port can pass.
3. Linux Bridge VLAN realization principle
For the physical device, a routing switch connected to set different access interface VLAN connect to the switch, provided the connection between the trunk port switches, directly connected to the physical machine switches, but the kvm virtual machine, as shown in FIG. , the physical NICs on the host virtual interfaces eth0.10, VLAN is a virtual interface device over a virtual virtual bridge interface, connected via the virtual machine virtual NIC vent0. The physical connection is different, kvm achieve VLAN must have the virtual network adapter and network
Similarly, for a plurality of virtual machines, multiple physical NICs to connect to VLAN.
4. Linux Bridge achieve VLAN
1> Check whether the core VLAN function
By dmesg command to view the physical configuration, to see if there are 802 fields, while viewing the / proc / net / vlan directory exists. If no VLAN function, the directory does not exist.
[root@localhost ~]# dmesg | grep -i 802 [ 0.380228] pci 0000:00:11.0: PCI bridge to [bus 02] (subtractive decode) [ 0.380255] pci 0000:00:11.0: bridge window [io 0x2000-0x3fff] [ 0.380281] pci 0000:00:11.0: bridge window [mem 0xfd500000-0xfdffffff] [ 0.580240] pci 0000:00:17.6: bridge window [io 0x1000-0x0fff] to [bus 19] add_size 1000 [ 1.221802] pcieport 0000:00:15.7: irq 32 for MSI/MSI-X [ 1.228025] pciehp 0000:00:18.5:pcie04: Slot #261 AttnBtn+ PwrCtrl+ MRL- AttnInd- PwrInd- HotPlug+ Surprise- Interlock- NoCompl+ LLActRep+ [ 1.249802] hp_sw: device handler registered [ 2.917802] systemd[1]: Inserted module 'ip_tables'
若8021模块没有载入系统。则可以使用 modprobe模块组命令载入8021q模块,再利用lsmod命令查看模块是否载入到核心内。
[root@localhost ~]# modprobe 8021q [root@localhost ~]# lsmod | grep 8021q 8021q 33104 0 garp 14384 1 8021q mrp 18542 1 8021q [root@localhost ~]#
设置开机载入8021q模块(可选)
在/etc/sysconfig/modules下增加一个8021q.modules文件,文件内容为modprobe 8021q
vim /etc/sysconfig/modules/8021q.modules
modprobe 8021q
2> 安装VLAN管理工具vconfig
[root@localhost ~]# rz
[root@localhost ~]# ls
vconfig-1.9-16.el7.x86_64.rpm [root@localhost ~]# yum localinstall vconfig-1.9-16.el7.x86_64.rpm -y
3> 创建VLAN接口
创建VLAN接口前,在设备上添加一块网卡,并配置为静态,在ens37这块网卡上设置VLAN。
添加网卡
[root@localhost ~]# cd /etc/sysconfig/network-scripts/ [root@localhost network-scripts]# ls [root@localhost network-scripts]# nmtui
修改网卡配置文件
[root@localhost ~]# cd /etc/sysconfig/network-scripts/ [root@localhost network-scripts]# vim ifcfg-ens37 BOOTPROTO=static #只改这一行即可 [root@localhost network-scripts]# systemctl restart network #由于无法获取ip,重启有可能起不来,由于该网卡只用作VLAN,起不来也可以 [root@localhost ~]# ip a .... 9: ens37: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:cc:05:06 brd ff:ff:ff:ff:ff:ff inet6 fe80::4d27:24e8:e097:435c/64 scope link valid_lft forever preferred_lft forever ......
添加VLAN。用命令vconfig add
[root@localhost ~]# vconfig add ens37 10 Added VLAN with VID == 10 to IF -:ens37:- [root@localhost ~]# ip a .. 11: ens37.10@ens37: <BROADCAST,MULTICAST,M-DOWN> mtu 1500 qdisc noop state DOWN qlen 1000 link/ether 00:0c:29:cc:05:06 brd ff:ff:ff:ff:ff:ff ..... [root@localhost ~]# vconfig add ens37 20 Added VLAN with VID == 20 to IF -:ens37:-
给两个VLAN端口进行配置,写入网桥brvlan
[root@localhost network-scripts]# cp ifcfg-ens37 ifcfg-ens37.10 [root@localhost network-scripts]# vim ifcfg-ens37.10 VLAN=yes TYPE=vlan PHYSDEV=ens37 VLAN_ID=10 NAME=ens37.10 ONBOOT=yes ZONE=trusted DEVICE=ens37.10 BRIDGE=brvlan-10 [root@localhost network-scripts]# cp ifcfg-ens37.10 ifcfg-ens37.20 [root@localhost network-scripts]# vim ifcfg-ens37.20 VLAN=yes TYPE=vlan PHYSDEV=ens37 VLAN_ID=10 NAME=ens37.10 ONBOOT=yes ZONE=trusted DEVICE=ens37.10 BRIDGE=brvlan-10 :%s/10/20/g 结果: VLAN=yes TYPE=vlan PHYSDEV=ens37 VLAN_ID=20 NAME=ens37.20 ONBOOT=yes ZONE=trusted DEVICE=ens37.20 BRIDGE=brvlan-20
添加两块网桥
[root@localhost ~]# brctl addbr brvlan-10 [root@localhost ~]# brctl addbr brvlan-20
配置网桥
[root@localhost network-scripts]# vim ifcfg-brvlan-10 TYPE=bridge BOOTPROTO=static NAME=brvlan-10 DEVICE=brvlan-10 ONBOOT=yes [root@localhost network-scripts]# vim ifcfg-brvlan-20 TYPE=bridge BOOTPROTO=static NAME=brvlan-20 DEVICE=brvlan-20 ONBOOT=yes
将网桥brvlan-10 接到网口ens37.10,brvlan-20 接到网口ens37.20,用命令brctl addif
[root@localhost network-scripts]# brctl addif brvlan-10 ens37.10 [root@localhost network-scripts]# brctl addif brvlan-20 ens37.20
重启,查看,
[root@localhost network-scripts]# systemctl stop NetworkManager [root@localhost network-scripts]# systemctl restart network [root@localhost network-scripts]# ip a 11: ens37.10@ens37: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master brvlan-10 state UP qlen 1000 link/ether 00:0c:29:cc:05:06 brd ff:ff:ff:ff:ff:ff 12: ens37.20@ens37: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master brvlan-20 state UP qlen 1000 link/ether 00:0c:29:cc:05:06 brd ff:ff:ff:ff:ff:ff 13: brvlan-10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP qlen 1000 link/ether 00:0c:29:cc:05:06 brd ff:ff:ff:ff:ff:ff inet6 fe80::20c:29ff:fecc:506/64 scope link valid_lft forever preferred_lft forever 14: brvlan-20: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP qlen 1000 link/ether 00:0c:29:cc:05:06 brd ff:ff:ff:ff:ff:ff inet6 fe80::20c:29ff:fecc:506/64 scope link valid_lft forever preferred_lft forever
4> 克隆虚拟机
实验会用到两台虚拟机,但由于宿主机192.168.16.3仅有一台虚拟机centos7.0,因此需要克隆一份虚拟再做实验。
克隆虚拟机的两种方式:图形界面直接克隆、代码界面利用命令 virsh -clone进行克隆。
1) 图形化界面克隆
将要克隆的虚拟机关机,右键进行克隆。
2) 命令行界面克隆
下载virt
[root@localhost ~]# mount /dev/cdrom /mnt mount: /dev/sr0 写保护,将以只读方式挂载 [root@localhost ~]# yum install virt* -y
克隆
[root@localhost ~]# virt-clone -o centos7.0 -n vm2 -f /var/lib/libvirt/vm2.qcow2 成功克隆 'vm2'。
查看
[root@localhost ~]# virsh list --all Id 名称 状态 ---------------------------------------------------- - centos7.0 关闭 - vm1 关闭 - vm2 关闭
5> 虚拟机连接网桥
VM1连brvlan-10,VM2连brvlan-20
6> 开启两台虚拟机。此时我们的框架已经搭好了。
7> 测试两台虚拟机是否能通信
为了能够测试,先手动给两个虚拟机设置ip
vm1:10.10.10.10 ;vm2:10.10.10.20
此时两台虚拟机是互不相通的,因为brvlan-10和brvlan-20不在同一个网段,将其改到以同一个网段再测试
若两台虚拟机想通外网,只需加一块网卡即可
由于在配置文件/etc/resove.conf含有域名解析,因此能访问百度。
5. 网卡配置bond(绑定)
1>
网卡bond(绑定),也称作网卡捆绑。就是将两个或者更多的物理网卡绑定成一个虚拟网卡。网卡是通过把多张网卡绑定为一个逻辑网卡,实现本地网卡的冗余,带宽扩容和负载均衡,在应用部署中是一种常用的技术。
多网卡绑定实际上需要提供一个额外的软件的bond驱动程序实现。通过驱动程序可以将多块网卡屏蔽。对TCP/IP协议层只存在一个Bond网卡,在Bond程序中实现网络流量的负载均衡,即将一个网络请求重定位到不同的网卡上,来提高总体网络的可用性。
2> 网卡绑定的目的
提高网卡的吞吐量。
增强网络的高可用,同时也能实现负载均衡。
3> 网卡配置bond(绑定)bond模式:
1)Mode=0(balance-rr) 表示负载分担round-robin,平衡轮询策略,具有负载平衡和容错功能;
bond的网卡MAC为当前活动的网卡的MAC地址(bond0),需要交换机设置聚合模式,将多个网卡绑定为一条链路。
2)Mode=1(active-backup) 表示主备模式,具有容错功能,只有一块网卡是active,另外一块是备的standby,这时如果交换机配的是捆绑,将不能正常工作,因为交换机往两块网卡发包,有一半包是丢弃的。
3) Mode=2(balance-xor) 表示XOR Hash负载分担(异或平衡策略),具有负载平衡和容错功能。每个slave接口传输每个数据包和交换机的聚合强制不协商方式配合。(需要xmit_hash_policy)。
4) Mode=3(broadcast) 表示所有包从所有interface发出,广播策略,具有容错能力,这个不均衡,只有冗余机制...和交换机的聚合强制不协商方式配合。
5)Mode=4(802.3ad) 表示支持802.3ad协议(IEEE802.3ad 动态链接聚合) 和交换机的聚合LACP方式配合(需要xmit_hash_policy)。
6)Mode=5(balance-tlb) 适配器传输负载均衡,并行发送,无法并行接收,解决了数据发送的瓶颈。 是根据每个slave的负载情况选择slave进行发送,接收时使用当前轮到的slave。
7) Mode=6(balance-alb) 在5的tlb基础上增加了rlb。适配器负载均衡模式并行发送,并行接收数据包。
5)和6)不需要交换机端的设置,网卡能自动聚合。4需要支持802.3ad。0,2和3理论上需要静态聚合方式,但实测中0可以通过mac地址欺骗的方式在交换机不设置的情况下不太均衡地进行接收。
常用的有三种:
mode=0:平衡负载模式,有自动备援,但需要”Switch”支援及设定。
mode=1:自动备援模式,其中一条线若断线,其他线路将会自动备援。
mode=6:平衡负载模式,有自动备援,不必”Switch”支援及设定。
4> 以模式6为例演示bond
实验需要两台宿主机,每台宿主机需要五张网卡。
1) 添加四块网卡,并克隆一台宿主机
网卡1设置为nat模式连外网,网卡2~5设置为vmnat1仅主机模式进行bond(绑定)
克隆虚拟机
2) 解绑实验环境
先将3. linux bridge实验的ens37网卡进行解绑
1)) 关闭虚拟机,从网桥脱离网口
[root@localhost ~]# brctl show bridge name bridge id STP enabled interfaces br0 8000.0050563d215c no ens33 brvlan-10 8000.000c2942c3f7 no ens37.10 brvlan-20 8000.000c2942c3f7 no ens37.20 virbr0 8000.5254009483b2 yes virbr0-nic [root@localhost ~]# brctl delif brvlan-10 ens37.10 [root@localhost ~]# brctl delif brvlan-20 ens37.20 [root@localhost ~]# brctl show bridge name bridge id STP enabled interfaces br0 8000.0050563d215c no ens33 brvlan-10 8000.000000000000 no brvlan-20 8000.000000000000 no
2))删除网桥
[root@localhost ~]# brctl delbr brvlan-10 bridge brvlan-10 is still up; can't delete it #命令行无法进行删除,使用图形化界面进行删除 [root@localhost ~]# nmtui
3)) 删除虚拟网口
[root@localhost ~]# vconfig rem ens37.10 Removed VLAN -:ens37.10:- [root@localhost ~]# vconfig rem ens37.20 Removed VLAN -:ens37.20:-
3)将ens37、38、39、40都配置绑定为bond0
[root@localhost ~]# cd /etc/sysconfig/network-scripts/ [root@localhost network-scripts]# vim ifcfg-ens37 TYPE=Ethernet BOOTPROTO=none DEVICE=ens37 ONBOOT=yes MASTER=bond0 SLAVE=yes
[root@localhost network-scripts]# vim ifcfg-ens38 TYPE=Ethernet BOOTPROTO=none DEVICE=ens38 ONBOOT=yes MASTER=bond0 SLAVE=yes [root@localhost network-scripts]# vim ifcfg-ens39 TYPE=Ethernet BOOTPROTO=none DEVICE=ens39 ONBOOT=yes MASTER=bond0 SLAVE=yes [root@localhost network-scripts]# vim ifcfg-ens40 TYPE=Ethernet BOOTPROTO=none DEVICE=ens40 ONBOOT=yes MASTER=bond0 SLAVE=yes
4) 加载bonding模块
[root@localhost network-scripts]# modprobe bonding
5) 创建bond0网卡配置文件
[root@localhost network-scripts]# vim ifcfg-bond0 DEVICE=bond0 TYPE=Bond NAME=bond0 BONDING_MASTER=yes #设置为 绑定网卡主 BOOTPROTO=static USERCTL=no ONBOOT=yes BONDING_OPTS="mode=6 miimon=100" #模式6,检测100秒网卡的状态 BRIDGE=br1
6) 配置网桥br1
[root@localhost network-scripts]# vim ifcfg-br1 TYPE=Bridge DEVICE=br1 ONBOOT=yes BOOTPROTO=static NAME=br1
7) 在br1上创建虚拟网口br1.10、br1.20
[root@localhost network-scripts]# brctl addbr br1 #添加虚拟网口 [root@localhost network-scripts]# brctl addbr br2 [root@localhost network-scripts]# vconfig add br1 10 #添加VLAN Added VLAN with VID == 10 to IF -:br1:- [root@localhost network-scripts]# vconfig add br1 20 Added VLAN with VID == 20 to IF -:br1:- [root@localhost network-scripts]# ip a 15: br1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000 link/ether 62:27:26:04:33:2f brd ff:ff:ff:ff:ff:ff 16: br2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000 link/ether 66:1c:df:79:22:3a brd ff:ff:ff:ff:ff:ff 17: br1.10@br1: <BROADCAST,MULTICAST,M-DOWN> mtu 1500 qdisc noop state DOWN qlen 1000 link/ether 62:27:26:04:33:2f brd ff:ff:ff:ff:ff:ff 18: br1.20@br1: <BROADCAST,MULTICAST,M-DOWN> mtu 1500 qdisc noop state DOWN qlen 1000 link/ether 62:27:26:04:33:2f brd ff:ff:ff:ff:ff:ff
8) 创建虚拟网桥brvlan-10、brvlan-20
[root@localhost network-scripts]# brctl addbr brvlan-10 [root@localhost network-scripts]# brctl addbr brvlan-20
9) 连接虚拟网桥和VLAN接口
[root@localhost network-scripts]# brctl addif brvlan-10 br1.10 [root@localhost network-scripts]# brctl addif brvlan-20 br1.20 [root@localhost network-scripts]# brctl show brvlan-10 8000.62272604332f no br1.10 brvlan-20 8000.62272604332f no br1.20
10) 配置虚拟网口
[root@localhost network-scripts]# vim ifcfg-br1.10 VLAN=yes TYPE=vlan PHYSDEV=br1 VLAN_ID=10 NAME=br1.10 ONBOOT=yes ZONE=trusted DEVICE=br1.10 BRIDGE=brvlan-10 [root@localhost network-scripts]# vim ifcfg-br1.20 VLAN=yes TYPE=vlan PHYSDEV=br1 VLAN_ID=20 NAME=br1.20 ONBOOT=yes ZONE=trusted DEVICE=br1.20 BRIDGE=brvlan-20
11) 配置虚拟网桥
[root@localhost network-scripts]# vim ifcfg-brvlan-10 TYPE=bridge BOOTPROTO=static NAME=brvlan-10 DEVICE=brvlan-10 ONBOOT=yes [root@localhost network-scripts]# vim ifcfg-brvlan-20 TYPE=bridge BOOTPROTO=static NAME=brvlan-20 DEVICE=brvlan-20 ONBOOT=yes
12) 重启网络
[root@localhost network-scripts]# systemctl restart network [root@localhost network-scripts]# ip a ....... 17: br1.10@br1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master brvlan-10 state UP qlen 1000 link/ether 62:27:26:04:33:2f brd ff:ff:ff:ff:ff:ff 18: br1.20@br1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master brvlan-20 state UP qlen 1000 link/ether 62:27:26:04:33:2f brd ff:ff:ff:ff:ff:ff
13) 打开虚拟机,连接测试
修改网卡连接
用vm1 ping vm2,由于vm1处在brvlan-10而vm2处在brvlan-20,不在同一网段,因此无法ping通。
如果将vm1、vm2设置为同一个VLAN,则两台虚拟机可以ping通