OpenStack PCI passthrough environment configuration

OpenStack PCI passthrough environment configuration

Author: Zhang Hangdong

OpenStack version: Kilo

Virtual machines use transparent devices to achieve near-native device performance. Both Intel and AMD offer support for device passthrough (and new instructions to assist the hypervisor) in their next-generation processor architectures. Intel calls this support Virtualization Technology for Directed I/O (VT-d) , while AMD calls it the I/O Memory Management Unit (IOMMU) . In either case, the latest CPUs provide a way to map PCI physical addresses to guest virtual systems. When this mapping occurs, the hardware is responsible for accessing (and protecting) the device, and the guest operating system uses the device as if it were not a virtual system. In addition to mapping clients to physical memory, the new architecture also provides isolation mechanisms to prevent other clients (or hypervisors) from accessing that memory in advance.

1. Confirm whether the Host supports pci-passthrough

Since hardware support is required, to confirm in advance whether the CPU and motherboard support the hardware-assisted virtualization function of Intel or AMD , please refer to the official hardware support list or check the relevant options in the BIOS .

In addition, the so-called clever woman is hard to cook without rice, and CPU support is only a necessary condition for pci-passthrough technology.

A network card that supports pci-passthrough is required.

The following takes Intel E5-2690 + Intel X540 10G NIC + RHEL7.0 as an example.

2. Check whether the Host has enabled the hardware-assisted virtualization function

[root@nova2 ~]# cat /proc/cmdline

BOOT_IMAGE=/vmlinuz-3.10.0-229.el7.x86_64

root=/dev/mapper/rhel-root

rd.lvm.lv=rhel/root

crashkernel=auto

rd.lvm.lv=rhel/swap

vconsole.font=latarcyrheb-sun16

vconsole.keymap=us

rhgb

quiet

LANG = en_US.UTF-8

intel_iommu=on #There is this field to indicate that the Intel hardware-assisted virtualization function is turned on

default_hugepagesz=1G hugepagesz=1G hugepages=20

If it is confirmed that the hardware supports the hardware-assisted virtualization function but it is not enabled, you can configure it as follows:

[root@nova2 ~] # vi /boot/grub2/grub.cfg #Different OS or versions may have different files

#Add the following to the startup parameters in it

intel_iommu=on

Then reboot to make the configuration take effect

3. Confirm NIC information

[root@nova2 ~]# lspci -nn | grep Ethernet

01:00.0 Ethernet controller [0200]: Intel Corporation I350 Gigabit Network Connection [8086:1521] (rev 01)

01:00.1 Ethernet controller [0200]: Intel Corporation I350 Gigabit Network Connection [8086:1521] (rev 01)

82:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 [8086:1528] (rev 01)

82:00.1 Ethernet controller [0200]: Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 [8086:1528] (rev 01)

84:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 [8086:1528] (rev 01)

84:00.1 Ethernet controller [0200]: Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 [8086:1528] (rev 01)

此处以Intel X540 10G NIC为例：

1.84:00.1 pci bus address

2.[8086 vendor id

3.1528] product id

(记住这三个信息，后面会用到)

4. 确认pci设备驱动信息并从host默认驱动程序中解绑，以备虚拟机透传使用

[root@nova2 ~]# virsh nodedev-list | grep pci | grep 84 #84是从上面的①来的

pci_0000_84_00_0

pci_0000_84_00_1

#因为这块网卡有两个物理port所以grep出来两条结果，下面我们只用pci_0000_84_00_1来演示

#然后确认pci_0000_84_00_1的相关信息

[root@nova2 ~]# virsh nodedev-dumpxml pci_0000_84_00_1

<path>/sys/devices/pci0000:80/0000:80:03.0/0000:84:00.1</path>

<name>ixgbe</name> #host默认驱动

</driver>

<product id='0x1528'>Ethernet Controller 10-Gigabit X540-AT2</product>

<vendor id='0x8086'>Intel Corporation</vendor>

</iommuGroup>

</capability>

</device>

#从host的默认驱动ixgbe中解绑pci_0000_84_00_1

[root@nova2 ~]# virsh nodedev-detach pci_0000_84_00_1

Device pci_0000_84_00_1 detached

#再次确认解绑后的pci_0000_84_00_1信息

[root@nova2 ~]# virsh nodedev-dumpxml pci_0000_84_00_1

<path>/sys/devices/pci0000:80/0000:80:03.0/0000:84:00.1</path>

<name>vfio-pci</name> #驱动程序变为vfio(目前默认的虚拟化驱动)

</driver>

<product id='0x1528'>Ethernet Controller 10-Gigabit X540-AT2</product>

<vendor id='0x8086'>Intel Corporation</vendor>

</iommuGroup>

</capability>

</device>

5. 配置openstack，以启用pci-passthrough：

#Nova controller

[root@osc ~]# vi /etc/nova/nova.conf

# Options defined in nova.pci.pci_request

# An alias for a PCI passthrough device requirement. This

# allows users to specify the alias in the extra_spec for a

# flavor, without needing to repeat all the PCI property

# requirements. For example: pci_alias = { "name":

# "QuicAssist", "product_id": "0443", "vendor_id": "8086",

# "device_type": "ACCEL" } defines an alias for the Intel

# QuickAssist card. (multi valued) (multi valued)

#pci_alias=

pci_alias={ "name":"X540NIC", "slabel":"dpdk-int"}

① ② ③ ④

①name字段，固定

②name值，可以自己随便起

③不知道什么意思，好像也是自己起的

④不知道什么意思，好像也是自己起的

#Nova compute

[root@osc ~]# vi /etc/nova/nova.conf

# Options defined in nova.pci.pci_whitelist

# White list of PCI devices available to VMs. For example:

# pci_passthrough_whitelist = [{"vendor_id": "8086",

# "product_id": "0443"}] (multi valued)

#pci_passthrough_whitelist=

pci_passthrough_whitelist={"vendor_id":"8086", "product_id":"1528", "slabel":"dpdk-int"}

⑤ ⑥ ⑦ ⑧ ⑨ ⑩

⑤vendor_id字段，固定

⑥vendor_id的值，对应于表3中的②

⑦product_id字段，固定

⑧product_id的值，对应于表3中的③

⑨不知道什么意思，但要和controller的配置保持一致

⑩不知道什么意思，但要和controller的配置保持一致

改完重启OpenStack controller和compute服务。

6. 创建带有pci-passthrough的flavor：

在flavor中添加metadata，字段名为”pci_passthrough:alias”，值分为两部分，第一部分为别名，和表5中的②保持一致，第二部分为数量，根据实际情况填写即可。

7. 创建虚拟机，并确认pci-passthrough设备分配至虚拟机

如果确认硬件支持硬件辅助虚拟化功能但没有开启，可以进行如配置： [root@nova2 ~]# vi /boot/grub2/grub.cfg #不同的OS或版本可能文件不同 #向其中的启动参数中添加以下

OpenStack PCI passthrough environment configuration

Guess you like