ovs-dpdk + qemu uses vhostuserclient to communicate, the connection is blocked, and libvirt reports the following error:
83 2018-05-04 13:17:37.265+0000: starting up libvirt version: 3.2.0, package: 14.el7_4.9 (CentOS BuildSystem <http://bugs.centos.org>, 2018-03-07-13:51:24, x86-01.bsys.centos.org), qemu version: 2.9.0, hostna 84 LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin QEMU_AUDIO_DRV=none /export/Datapath/tong/qemu_home/bin/qemu-system-x86_64 -name guest=centos7.0,debug-threads=on -S -object secret,id=master 85 2018-05-04T13:17:37.295614Z qemu-system-x86_64: -chardev socket,id=charnet1,path=/tmp/nlb_vm0.sock,server: QEMU waiting for connection on: disconnected:unix:/tmp/nlb_vm0.sock,server 86 2018-05-04T13:17:37.808975Z qemu-system-x86_64: -chardev socket,id=charnet2,path=/tmp/nlb_vm1.sock,server: QEMU waiting for connection on: disconnected:unix:/tmp/nlb_vm1.sock,server 87 2018-05-04T13:17:38.809417Z qemu-system-x86_64: -chardev pty,id=charserial0: char device redirected to /dev/pts/3 (label charserial0) 88 2018-05-04T13:17:45.706248Z qemu-system-x86_64: Failed initializing vhost-user memory map, consider using -object memory-backend-file share=on 89 2018-05-04T13:17:45.706284Z qemu-system-x86_64: vhost_set_mem_table failed: Interrupted system call (4) 90 2018-05-04T13:17:45.706421Z qemu-system-x86_64: unable to start vhost net: 4: falling back on userspace virtio 91 2018-05-04T13:17:45.708417Z qemu-system-x86_64: Failed initializing vhost-user memory map, consider using -object memory-backend-file share=on 92 2018-05-04T13:17:45.708436Z qemu-system-x86_64: vhost_set_mem_table failed: Resource temporarily unavailable (11) 93 2018-05-04T13:17:45.708558Z qemu-system-x86_64: unable to start vhost net: 11: falling back on userspace virtio 94 2018-05-04T13:33:01.734457Z qemu-system-x86_64: terminating on signal 15 from pid 2367 (/usr/sbin/libvirtd)
Add a line to libvirt's xml:
20 <cpu mode='host-passthrough' check='none'> 21 <topology sockets='1' cores='4' threads='2'/> 22 <numa> 23 <cell id='0' cpus='0-7' memory='16777216' unit='KiB' memAccess='shared'/> 24 </numa> 25 </cpu>
The corresponding qemu command has been added:
-object memory-backend-file,id=ram-node0,prealloc=yes,mem-path=/dev/hugepages/libvirt/qemu/6-centos7.0,share=yes,size=17179869184 -
Troubleshooting succeeded.
refer to:
http://docs.openvswitch.org/en/latest/topics/dpdk/vhost-user/
In addition, QEMU must allocate the VM’s memory on hugetlbfs. vhost-user ports access a virtio-net device’s
virtual rings and packet buffers mapping the VM’s physical memory on hugetlbfs. To enable vhost-user ports
to map the VM’s memory into their process address space, pass the following parameters to QEMU:
- object memory-backend-file,id=mem,size=4096M,mem-path=/dev/hugepages,share= on -numa node,memdev=mem -mem-prealloc
https://libvirt.org/formatdomain.html#elementsNUMATuning