As we mentioned in "KVM Architecture of KVM Virtualization Solution Series", the KVM module, as the core of the entire virtualization environment, works in the kernel space and is responsible for CPU and memory scheduling. QEMU works as an emulator in user space and is responsible for virtual machine I/O simulation. The creation and operation of a KVM virtual machine is a process in which the kernel-mode KVM module and the user-mode QEMU program cooperate with each other. This actually shows that the KVM virtualization environment needs to include the kernel-mode KVM module and the user-mode QEMU program. It is very important to understand this because it is the basis of the entire KVM virtualization.
KVM虚拟化=内核态KVM模块+用户态QEMU程序
After the host operating system is built, this chapter really enters the deployment of the KVM virtualization environment. Next, I plan to do so. I plan to use a table to briefly introduce the requirements for building a KVM environment under the CentOS system and the Ubuntu system. software, tools and steps, and then introduce the construction, as shown in Figure 1.
According to Figure 1, we can understand that there are two ways to build a KVM virtualization environment, whether it is a CentOS system or an Ubuntu system. Below I will make a simple convergence on the content of Figure 1.
In CentOS system
In mode 1, KVM virtualization is a combination of a kvm kernel module and a qemu-kvm user program (qemu-kvm is a modified QEMU), where qemu-kvm does not have the qemu-system-x86_64 tool, that is to say If method 1 is used to build KVM virtualization, only qemu-img is the tool for creating virtual machine disks, and there is still a lack of tools for creating virtual machines. You also need to install the virt-install tool based on the libvirt suite to create a virtual machine. The content of the libvirt suite will be introduced in subsequent chapters, so I won’t go into details here.
In method 2, KVM virtualization is a combination of a kvm kernel module and a Qemu virtualization software. Among them, the Qemu virtual software comes with qemu-img and qemu-system-x86_64 tools. But here comes the problem again, Qemu is a virtualization software independent of KVM, and is not part of KVM, so directly using Qemu-system-x86_64 that comes with Qemu to create a virtual machine will result in particularly low efficiency, because Qemu is a pure software virtualization technology. But it doesn't matter, this problem has been solved at the technical level. Before 2012, qemu-kvm was a QEMU branch dedicated to KVM. In 2012, this branch was merged into the mainstream QEMU. After that, it will be implemented KVM virtualization does not require a special qemu-kvm (you can continue to use it, and CentOS is now using it), but only needs to add the "-enable-kvm" option to the general QEMU command, which is equivalent to using the KVM function Yes, KVM drives the virtual machine, that is, QEMU can call the KVM kernel, so the efficiency is higher than Qemu, the command line is listed below.
[root@kvm01 ~]# qemu-system-x86_64 -enable-kvm -m 6G /root/lisq.img # 选择KVM加速
In Ubuntu system
In mode 1, KVM virtualization is a combination of a kvm kernel module and a qemu-kvm user program (qemu-kvm is a modified QEMU), where qemu-kvm comes with qemu-im and qemu-system-x86_64 tools , you can directly use method 1 to build a KVM virtualization environment. Note that in qemu-kvm, the qemu-system-x86_64 tool still needs to add the "-enable-kvm" option to use KVM to drive the virtual machine.
In method 2, this part of the content refers to the content of method 2 of the CentOS system. There is no difference between method 1 and method 2. The qemu-system-x86_64 tool needs to add the "-enable-kvm" option to use KMV to drive the virtual machine.
3. KVM installation
Starting from version 2.6.20 of Linux, KVM has been fully and officially included in the Linux kernel. KVM exists as a module in the Linux kernel. Therefore, as long as the version of Linux 2.6.20 or higher than Linux 2.6.20 is used, the When the system is installed, the KVM module has already been loaded, so we only need to enable the KVM module in the CLI command line mode.
3.1. Install KVM on CentOS
The first step is to check whether the KVM module is loaded
By default, the CentOS 7 system is loaded with the KVM module. You can use the command "lsmod | grep kvm" to check whether the KVM module is loaded. If it is loaded, there is no need to start the KVM module. If it is not loaded, you need to perform the second step.
[root@kvm01 ~]# lsmod | grep kvm # 默认情况下，已经加载了KVM模块
kvm_intel 188740 0
kvm 637515 1 kvm_intel
irqbypass 13503 1 kvm
The second step is to enable the KVM module
[root@kvm01 ~]# modprobe kvm # 加载kvm模块
[root@kvm01 ~]# modprobe kvm-intel # 如果是AMD加载，加载kvm-amd
[root@kvm01 ~]# lsmod | grep kvm
kvm_intel 188740 0
kvm 637515 1 kvm_intel
irqbypass 13503 1 kvm
In the third step, KVM is loaded successfully, and the /dev/kvm device interface will be created
[root@kvm01 ~]# ll /dev/kvm
crw-rw-rw-+ 1 root kvm 10, 232 10月 18 10:51 /dev/kvm
3.2. Install KVM on Ubuntu
The first step is to check whether the KVM module is loaded
By default, the Ubuntu system also loads the KVM module. You can use the command "lsmod | grep kvm" to check whether the KVM module is loaded. If it is loaded, there is no need to start the KVM module. If it is not loaded, you need to perform the second step.
root@kvm02:~# lsmod | grep kvm # 默认情况下，已经加载了KVM模块
kvm_intel 294912 0
kvm 823296 1 kvm_intel
The second step is to enable the KVM module
root@kvm02:~# modprobe kvm # 加载kvm模块
root@kvm02:~# modprobe kvm-intel # 如果是AMD加载，加载kvm-amd
root@kvm02:~# lsmod | grep kvm
kvm_intel 294912 0
kvm 823296 1 kvm_intel
In the third step, KVM is loaded successfully, and the /dev/kvm device interface will be created
root@kvm02:~# ls -l /dev/kvm
crw-rw----+ 1 root kvm 10, 232 10月 18 11:32 /dev/kvm
4. QEMU installation
4.1. Install QEMU on CentOS
In order to obtain qemu-system-x86_64, use method 2 to realize KVM virtualization, and QEMU uses the source code installation method. Note that under CentOS, QEMU pure software cannot be installed by YUM, but can only be installed by source code. Using source code to install will encounter many pitfalls. Different qemu versions and different CentOS versions have different pitfalls. Whoever tries will know.
The first step, QEMU source code download
[root@kvm01 ~]# yum install qemu -y # 使用yum方式安装发现没有rpm包，使用源码安装方式
[root@kvm01 ~]# wget https://download.qemu.org/qemu-4.0.0.tar.bz2 #下载源代码
[root@kvm01 ~]# tar xjvf qemu-4.0.0.tar.bz2 #解压源代码
The second step is to install the compilation tool
These compilation tools will be determined by the qemu version and the CentOS version. Different versions install different compilation tools. You can use the check command "./configue" to check, and install any missing compilation tools. I hope that all the tools missing in your version environment can be installed using yum, otherwise you will take a lot of pitfalls.
[root@kvm01 ~]# yum -y install gcc
[root@kvm01 ~]# yum -y install glib2-devel
[root@kvm01 ~]# yum -y install zlib-devel
[root@kvm01 ~]# yum -y install pixman-devel
The third step, check the dependencies
[root@kvm01 ~]# cd qemu-4.0.0/
[root@kvm01 qemu-4.0.0]#./configure # 检查当前的环境是否满足要安装软件的依赖关系
After satisfying the dependencies of installing the qemu-4.0.0 software, the following content is displayed:
[root@kvm01 qemu-4.0.0]# ./configure
No C++ compiler available; disabling C++ specific optional code
Install prefix /usr/local
BIOS directory /usr/local/share/qemu
firmware path /usr/local/share/qemu-firmware
binary directory /usr/local/bin
library directory /usr/local/lib
module directory /usr/local/lib/qemu
libexec directory /usr/local/libexec
include directory /usr/local/include
config directory /usr/local/etc
local state directory /usr/local/var
Manual directory /usr/local/share/man
ELF interp prefix /usr/gnemul/qemu-%M
Source path /root/qemu-4.0.0
GIT binary git
GIT submodules
C compiler cc
Host C compiler cc
C++ compiler
Objective-C compiler cc
ARFLAGS rv
CFLAGS -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -g
QEMU_CFLAGS -I/usr/include/pixman-1 -I$(SRC_PATH)/dtc/libfdt -pthread -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -fPIE -DPIE -m64 -mcx16 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv -std=gnu99 -Wendif-labels -Wno-missing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration -Wold-style-definition -Wtype-limits -fstack-protector-strong -Wno-missing-braces -I$(SRC_PATH)/capstone/include
LDFLAGS -Wl,--warn-common -Wl,-z,relro -Wl,-z,now -pie -m64 -g
QEMU_LDFLAGS -L$(BUILD_DIR)/dtc/libfdt
make make
install install
python python -B (2.7.5)
slirp support internal
smbd /usr/sbin/smbd
module support no
host CPU x86_64
host big endian no
target list aarch64-softmmu alpha-softmmu arm-softmmu cris-softmmu hppa-softmmu i386-softmmu lm32-softmmu m68k-softmmu microblazeel-softmmu microblaze-softmmu mips64el-softmmu mips64-softmmu mipsel-softmmu mips-softmmu moxie-softmmu nios2-softmmu or1k-softmmu ppc64-softmmu ppc-softmmu riscv32-softmmu riscv64-softmmu s390x-softmmu sh4eb-softmmu sh4-softmmu sparc64-softmmu sparc-softmmu tricore-softmmu unicore32-softmmu x86_64-softmmu xtensaeb-softmmu xtensa-softmmu aarch64_be-linux-user aarch64-linux-user alpha-linux-user armeb-linux-user arm-linux-user cris-linux-user hppa-linux-user i386-linux-user m68k-linux-user microblazeel-linux-user microblaze-linux-user mips64el-linux-user mips64-linux-user mipsel-linux-user mips-linux-user mipsn32el-linux-user mipsn32-linux-user nios2-linux-user or1k-linux-user ppc64abi32-linux-user ppc64le-linux-user ppc64-linux-user ppc-linux-user riscv32-linux-user riscv64-linux-user s390x-linux-user sh4eb-linux-user sh4-linux-user sparc32plus-linux-user sparc64-linux-user sparc-linux-user tilegx-linux-user x86_64-linux-user xtensaeb-linux-user xtensa-linux-user
gprof enabled no
sparse enabled no
strip binaries yes
profiler no
static build no
SDL support no
SDL image support no
GTK support no
GTK GL support no
VTE support no
TLS priority NORMAL
GNUTLS support no
libgcrypt no
nettle no
libtasn1 no
PAM no
iconv support yes
curses support no
virgl support no
curl support no
mingw32 support no
Audio drivers oss
Block whitelist (rw)
Block whitelist (ro)
VirtFS support no
Multipath support no
VNC support yes
VNC SASL support no
VNC JPEG support no
VNC PNG support no
xen support no
brlapi support no
bluez support no
Documentation no
PIE yes
vde support no
netmap support no
Linux AIO support no
ATTR/XATTR support yes
Install blobs yes
KVM support yes
HAX support no
HVF support no
WHPX support no
TCG support yes
TCG debug enabled no
TCG interpreter no
malloc trim support yes
RDMA support no
PVRDMA support no
fdt support git
membarrier no
preadv support yes
fdatasync yes
madvise yes
posix_madvise yes
posix_memalign yes
libcap-ng support no
vhost-net support yes
vhost-crypto support yes
vhost-scsi support yes
vhost-vsock support yes
vhost-user support yes
Trace backends log
spice support no
rbd support no
xfsctl support no
smartcard support no
libusb no
usb net redir no
OpenGL support no
OpenGL dmabufs no
libiscsi support no
libnfs support no
build guest agent yes
QGA VSS support no
QGA w32 disk info no
QGA MSI support no
seccomp support no
coroutine backend ucontext
coroutine pool yes
debug stack usage no
mutex debugging no
crypto afalg no
GlusterFS support no
gcov gcov
gcov enabled no
TPM support yes
libssh2 support no
TPM passthrough
TPM emulator
QOM debugging yes
Live block migration yes
lzo support no
snappy support no
bzip2 support no
lzfse support no
NUMA host support no
libxml2 no
tcmalloc support no
jemalloc support no
avx2 optimization yes
replication support yes
VxHS block device no
bochs support yes
cloop support yes
dmg support yes
qcow v1 support yes
vdi support yes
vvfat support yes
qed support yes
parallels support yes
sheepdog support yes
capstone internal
docker no
libpmem support no
libudev no
default devices yes
NOTE: cross-compilers enabled: 'cc' 'cc'
The fourth step, compile the source code
[root@kvm01 qemu-4.0.0]# make # 要点时间，慢慢等
The fifth step, install the source code
[root@kvm01 qemu-4.0.0]# make install
After the installation is complete, since QEMU is a program in user space, you can use qemu-mig and qemu-system-x86_64 directly without restarting the system after installation.
4.2. Install QEMU on Ubuntu
According to Figure 1, we can know that on the Ubuntu system, qemu-img and qemu-system-x86_64 can be obtained by installing qemu-kvm in YUM mode, which is very convenient. This article adopts this installation method. Of course, you can also use YUM to directly install the qemu suite to obtain qemu-img and qemu-system-x86_64. It is not recommended to use the source code to install the qemu suite.
The first step, YUM installs the qemu-kvm component
root@kvm02:~# apt-get install qemu-kvm # 安装qemu-kvm组件
After the installation is complete, since QEMU is a program in user space, you can use qemu-mig and qemu-system-x86_64 directly without restarting the system after installation. Note that although the qemu-system-x86_64 tool here comes with qemu-kvm, you still need to add the "-enable-kvm" option to enable KVM to drive the virtual machine when using this command.
So far, the KVM deployment of CentOS and Ubuntu operating systems is completed, and then we use the two tools of QEMU, qemu-img and qemu-system-x86_64, to create virtual machines.
5. Install the virtual machine
Here, we use the two tools qemu-img and qemu-system-x86_64 to create and start the virtual machine, but after these two tools, we need to write a long list of parameters, which is very unfriendly, and qemu-system-x86_64 The x86_64 tool is very inefficient, so most people will use other tools to manage virtual machines, such as the libvirt-based virt-install tool, etc. Tools such as libvirt will be introduced in subsequent chapters.
Before installing the virtual machine, we need to create an image file or disk partition to store the system and files in the virtual machine, which uses the qemu-img tool.
5.1. Install a virtual machine on CentOS
The first step is to create a virtual machine image file
Use the command "qemu-img" to create a blank virtual machine image, the format is raw, the image name is lisq.img, and the image size is 20G, as shown below:
[root@kvm01 ~]# qemu-img create -f raw lisq.img 20G
Formatting 'lisq.img', fmt=raw size=21474836480
The -f option is used to specify the image format, there are two formats of raw and qcow2, and even VMware's vmdk, vdi, vhd and other formats. The qcow2 format is the most commonly used image format for Qemu. qcow2 uses the copy-on-write technology to optimize performance. Here I use the raw format. lisq.img is the name of the image file, and 20G is the size of the image file.
Use the command "ls" to view the information of the lisq.img image, as shown below:
[root@kvm01 ~]# ls -lh lisq.img # 分配了20G空间
-rw-r--r--. 1 root root 20G 10月 19 14:51 lisq.img
Use the command "du -h" to view the space used by the lisq.img image, as follows:
[root@kvm01 ~]# du -h lisq.img # 虽然分配了20G空间，但是当前还没有使用
0 lisq.img
The second step is to upload the operating system
Install the WinSCP tool on the local computer. WinSCP uses port 22 to upload, so you need to ensure that the SSH service of the host kvm01 is enabled remotely, as shown in Figure 2.
After successful login, the interface shown in Figure 3 will be displayed. The left window of the interface is the window of the local computer, and the right window is the window of the host machine kvm01.
In the local computer window on the left, select an operating system image, drag it directly to the host kvm01 window on the right, and upload the operating system image to the host kvm01, as shown in Figure 4.
After the upload is complete, we will see the uploaded operating system ISO in the right host kvm01 window, which can also be viewed through "ls", as shown below:
[root@kvm01 ~]# ll
总用量 3073856
-rw-------. 1 root root 1601 10月 15 17:56 anaconda-ks.cfg
-rw-r--r--. 1 root root 1649 10月 15 18:02 initial-setup-ks.cfg
-rw-r--r--. 1 root root 21474836480 10月 19 14:51 lisq.img
drwxr-xr-x. 113 lisq lisq 12288 10月 19 13:43 qemu-4.0.0
-rw-r--r--. 1 root root 75668251 4月 24 2019 qemu-4.0.0.tar.bz2
-rw-r--r--. 1 root root 3071934464 9月 8 09:38 ubuntu-20.04.3-desktop-amd64.iso
The third step is to create a virtual machine
Use the command "qemu-system-x86_64" to create a virtual machine and install the operating system ISO just uploaded, as follows:
[root@kvm01 ~]# qemu-system-x86_64 -enable-kvm -m 6G -smp 4 -boot once=d -cdrom ubuntu-20.04.3-desktop-amd64.iso lisq.img # -enable-kvm是选择kvm模块，加速虚拟化效率
WARNING: Image format was not specified for 'lisq.img' and probing guessed raw.
Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.
Specify the 'raw' format explicitly to remove the restrictions.
VNC server running on ::1:5900
-m specifies the memory size of the virtual machine, the default unit is MB, -enable-kvm uses KVM for acceleration, -smp 4 specifies that the virtual machine is a symmetric multiprocessor structure and allocates 4 vCPUs, -boot once=d specifies the system The boot sequence is the first CD-ROM drive, and the default startup item (hard disk) will be used later. -cdrom is to allocate the virtual machine CD-ROM drive. Friendly reminder, it will be uncomfortable if the memory allocation of the virtual machine is less than 6G, 6G is the bottom line, otherwise this kind of experiment based on VMware virtual machine simulation host kvm01 cannot be done.
According to the output information, by default, QEMU will start a VNC server port (5900), and you can use the vncviwer tool to connect to the VNC port of QEMU to view the virtual machine. That is to say, the host kvm01 needs to install the software packages of vncserver and vncviewer tools.
Since we have used the qemu-system-x86_64 command to start the virtual machine, we cannot stop it if we want to keep the startup status, so we can also enable an Xshell login interface to install the two RPM packages tigervnc-server and tigervnc, as follows Show:
[root@kvm01 ~]# yum install tigervnc-server
[root@kvm01 ~]# yum install tigervnc
The fourth step, connect to the QEMU startup window
When prompted at startup, you can use the "vncviewer:5900" command to connect to the window started by QEMU.
Local connection method:
Open the terminal on the desktop of the host machine kvm01, and use the "vncviewer:5900" command to connect to the window started by QEMU, as shown in Figure 5.
Remote way:
If you don’t want to use the local host to connect to the QEMU window, but want to use the local computer to remotely connect to the QEMU window, you can do this, and open an Xshell interface to log in to the kmv01 host, and use the command "vncviewer: 5900" to connect to The window started by QEMU is as shown below.
[root@kvm01 ~]# vncviewer :5900
At this time, the following dialog box pops up, telling you that you need to install Xmanager software if you want to use this method, click the "Yes" button, and then go to the Xmanager official website to download the trial software according to legal requirements, as shown in Figure 6.
The Xmanager trial version software is installed on the local computer. After the installation is complete, you can log in remotely, so I won’t demonstrate it here.
Note that when you use the VMware virtual machine to simulate the host kvm, it is not recommended that you use the remote method to connect to QEMU, which is extremely inefficient.
The fifth step, install the operating system
After connecting to the QEMU window, the virtual machine starts to install the operating system, as shown in Figure 7.
The efficiency of QEMU's virtual machine is too low. Even if the "-enable-kvm" option is added to the qemu-system-x86_64 tool, even if the virtual machine is allocated 6G memory, the efficiency is still low. Press Crtl+C to cancel the installation detection and speed up the installation, as shown in Figure 8.
The virtual machine operating system is being installed, as shown in Figure 9. The installation process in the middle is omitted, and the efficiency of the entire installation process is very low, which takes about three and a half hours.
The installation process is very long. Even if the "-enable-kvm" option is added to the qemu-system-x86_64 tool, even if the virtual machine is allocated 6G of memory, the disk is also large enough. I feel that this QEMU may not use KVM hardware. Acceleration leads to very low efficiency. After installing a complete operating system, it takes three or four hours to get it right. In order to verify this idea, on the QEMU window, press the key combination "Ctrl+Alt+2" to switch to the QEMU monitor window, and execute the "info kvm" command to check whether the current QEMU uses KVM, as shown in Figure 10.
The result shows that kvm has been turned on, and QEMU is using KVM, which is displayed as "kvm support: enable", because we have connected -enable-kvm in the execution command. That can only show that the efficiency of using the qemu-system management tool is very low, and it can only explain this problem. In addition, it also explains a problem. I use a virtual machine environment to demonstrate the host machine, and the virtual machine nests the virtual machine, which is even more inefficient. Therefore, friends who have the conditions are best to use a physical host to build the environment.
The sixth step, restart the virtual machine system
After the system installation is complete, the virtual machine needs to be restarted to use the newly installed system, as shown in Figure 11.
Like the normal Linux system installation, after the installation is complete, restart the system to enter the virtual machine operating system just installed, as shown in Figure 12, after removing the installation media, press the Enter key to start the operating system from the hard disk .
We go back to the Xshell interface of the kvm01 host and delete or remove the operating system we uploaded, as shown below.
[root@kvm01 ~]# rm -rf ubuntu-20.04.3-desktop-amd64.iso
Then return to the interface in Figure 11, press the Enter key to continue restarting, and you can see that the system starts from the hard disk, as shown in Figure 13. The reason why I asked to remove the installation media is because I chose the "-boot once=d" command when creating the virtual machine. If I don't use this command, it will start directly from the hard disk.
The startup process is too slow. The virtual machine has 6G memory, and it takes about half an hour to start. After vomiting blood, the virtual machine operating system finally starts successfully, as shown in Figure 14.
After the virtual machine starts successfully, we log in to the virtual machine, and the login is successful, as shown in Figure 15.
The seventh step, start the virtual machine
After installing the virtual machine system, you can use the image file to start and log in to the system you installed. You can use the following command to start a virtual machine, as shown below:
[root@kvm01 ~]# qemu-system-x86_64 -enable-kvm -m 6G -smp 4 /root/lisq.img
VNC server running on ::1:5900
Use the command "vncviwer:5900" on the host machine kvm01 to connect to the QEMU window to check the startup status of the virtual machine.