Some summaries of knowledge about volumes/partitions in Linux

write in front

---

• Share some related knowledge Linuxwith your friends存储
• This part of things is basically useless, but there must be a 清晰definition
• The article covers:
  • Linuxfile storage, LVMrelated concepts
  • using the Linxufile system 格式化分区,分区挂载
  • Adjustment 逻辑卷, addition 交换分区, etc. actual combat Demo
  • The following concepts can be distinguished
  • 主分区, 扩展分区, 逻辑分区, 交换分区, 卷组, 物理卷, 逻辑卷, VDO卷etc.

Suicide is not necessarily weakness, it is often a firm protest, a helpless and tragic way for a fresh and lovely heart to demand meaning from life. ------- Shi Tiesheng "Me and the Temple of Earth"

---

1. Standard partition file storage

​​​​​

When we normally get a disk, we will use the lsblkview partition, and then we will use it fdiskor partedproceed 分区, and 分区then we will use the mkfsrelated 指定文件系统process 格式化处理. Finally, by mountproceeding 挂载处理and mounting it to the corresponding directory, we can use the directory normally. The order is as follows:

1. Identify the disk (lsblk)
2. partition
   • fdisk (MSDOS partition table)
   • gdisk (GPT partition table > 2.2TB disk)
   • parted (both possible)
3. Format (mkfs related)
4. mount
5. access mount point

specific process

$lsblk    #查看分区状态
$fdisk /dev/vdc     #使用fdisk分区工具，对/dev/vdc硬盘进行分区
p 查询分区状态 			
q 不保存退出
n 新建分区
Select (default p):      #回车，默认是主分区
Partition number (1-4, default 1):     #第一个分区默认序号是1
First sector ：    #起始扇区，直接回车
Last sector       +500M      #创建500M分区
w    #保存退出
$partprobe /dev/vdb 或者 partx -a /dev/vdb  #刷新硬盘分区表：
$mkfs.xfs /dev/vdc1     #使用xfs文件系统为vdc1这个分区格式化
$mount   /dev/vdc1   /mnt    #将新分区挂载到/mnt目录
$df -h   #h查看挂载信息
$umount   /dev/vdc1   #卸载分区

The following is the storage distribution of my machine. You can see that there is only one disk with two partitions, one ordinary partition, mounted to the root directory, and one swap partition.

┌──[[email protected]]-[~]
└─$ lsblk
NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda      8:0    0  200G  0 disk
├─sda1   8:1    0  150G  0 part /
└─sda2   8:2    0   10G  0 part [SWAP]
┌──[[email protected]]-[~]
└─$

Second, use LVM file storage

We created the filesystem on the hard drive using the standard partitions above, and adding extra space to an existing filesystem was somewhat of a pain. We can only resize partitions within the free space of the same physical hard disk.

Using Linux 逻辑卷管理器(logical volume manager，LVM)it allows you to easily manage disk space without rebuilding the entire file system.

What is LVM

One can be formed by groupingLVM together . would be considered , but in fact may be by . provides one , and these are .多个物理卷(物理分区)卷组(volume group，VG)逻辑卷管理系统卷组一个物理硬盘卷组分布在多个物理硬盘上的多个物理分区组成的卷组创建逻辑分区的平台逻辑分区包含了文件系统

The last layer in the whole structure is 逻辑卷(logical volume，LV). 逻辑卷Created for Linux 文件系统, 分区环境it works similarly to what we've been talking about so far in Linux 物理硬盘分区. Linux systems will be 逻辑卷treated as 物理分区. You can use either one , and then add it to 标准Linux文件系统a mount point in the catalog.格式化逻辑卷Linux虚拟目

The idea of ​​LVM logical volume management mechanism: 化零(物理卷PV)为整(卷组VG), 动态扩容伸缩, 按需(逻辑卷LV)分配. Reorganize scattered partitions (PV physical devices (physical volumes)) into large volume groups (VG virtual disks), and then obtain space as needed (virtual partition LV)

Links between physical volumes, logical volumes, volume groups, and snapshot volumes

• Physical Volume (PV) : refers to the hard disk partition, it can also be the entire hard disk or the created soft RAID, which is the basic storage device of LVM.
• Volume Group (VG) : It is a storage pool composed of one or more physical volumes, and one or more logical volumes can be created on the volume group.
• Logical Volume (LV) : Similar to a hard disk partition in a non-LVM system, it is built on a volume group and is a standard block device, and a file system can be built on top of a logical volume.
• Snapshots While the original Linux LVM allowed you to copy a logical volume to another device while it was online, traditional backup methods typically locked files while copying them to backup media. Snapshots allow you to replicate while keeping mission critical.

You can make such an assumption to understand the relationship between the above three: if you compare it PVto a plate of the earth, VGit is an earth, because the earth is composed of multiple plates, then divide an area on the earth and mark it as Asia, then Asia is equivalent to one LV.

Interconnection: When creating a volume group, be sure to reserve space for the snapshot of the logical volume, and then the snapshot accesses another entry of the logical volume. As long as the physical volume is added to the volume group, the physical space provided by the physical volume will be reserved in advance. It is divided into blocks, and this block is called PE (Physical Extend) [physical extent] before it is formatted. It is a small box of logical storage. The size of the volume group is composed of multiple PEs, and the size of the logical volume is The size is to put the PE in the volume group into the logical volume. At this time, the PE is no longer called PE, but is called LE (Logical Extend) [logical extent]. In fact, the LE in the logical volume is also called PE, which is just a station It's just a different angle.

If a physical volume is damaged, the LE stored in the logical volume will also be damaged. If you want the data not to be damaged, you can make a mirror of the PE in the physical volume. The mirror is a complete copy of the logical volume that is updated in real time. When you create a mirrored logical volume, LVM synchronizes the original logical volume into the mirrored copy

LVM management tools

• Physical volume operations (not commonly used): pvscan, pvdisplay, pvcreate
• Volume group operations: vgscan (scan), vgdisplay (display), vgcreate, vgremove, vgextend (expansion)
• Logical volume operations: lvscan, lvdisplay, lvcreate, lvremove, lvextend (PE: when allocating logical volume space, the smallest unit, the default is 4M)

Create a volume group:

Before creating a logical volume, create a volume group, and then divide the space from the volume group to the logical volume. Syntax:

$vgcreate  [-s  PE大小]  卷组名  分区.. ..

demo

$vgcreate /dev/myvg /dev/vdb2     #正常创建卷组，默认扩展单元是4M
$vgdisplay  /dev/myvg   #可以在PE一栏中看到扩张单元的大小
$vgremove /dev/myvg   #删除卷组myvg
$vgcreate -s 16MiB /dev/myvg /dev/vdb2    #创建扩展单元为16MiB的卷组（目前练习题要求），之后可以用vgdisplay查看PE的大小

Extend volume group

  vgextend  卷组名  分区.. ..

Create a logical volume:

grammar

$lvcreate  -L  大小  -n  名称   卷组名
$lvcreate  -l  PE个数  -n  名称   卷组名

demo

$lvcreate -n mylv -L 800M /dev/myvg    #正常创建
$lvcreate -n mylv -l 50 /dev/myvg    #按照扩展单元的数量创建之后可以用vgs查看
$vgs     #查看逻辑

Extend a logical volume:

$lvextend -L 300M /dev/test/vo    #将名字叫vo的逻辑卷扩容到300M

$blkid /dev/test/vo    #查看vo逻辑卷的文件系统
$lsblk    #再查看该逻辑卷的挂载点
$xfs_growfs /vo    #刷新大小，如果是xfs的文件系统
$resize2fs  /vo     #刷新大小，如果是ext的文件系统

The device location of the logical volume

/dev/卷组名/逻辑卷名
或者
/dev/mapper/卷组名-逻辑卷名

Scan Disk LVM Information

$vgscan 
  Reading all physical volumes.  This may take a while...
  Found volume group "myvg" using metadata type lvm2
  Found volume group "test" using metadata type lvm2
  Found volume group "rhel" using metadata type lvm2
$pvscan
  PV /dev/vdb2   VG myvg            lvm2 [1008.00 MiB / 208.00 MiB free]
  PV /dev/vdb1   VG test            lvm2 [<2.00 GiB / <1.67 GiB free]
  PV /dev/vda2   VG rhel            lvm2 [<29.00 GiB / 0    free]
  Total: 3 [<31.98 GiB] / in use: 3 [<31.98 GiB] / in no VG: 0 [0   ]
$lvscan 
  ACTIVE            '/dev/myvg/mylv' [800.00 MiB] inherit
  ACTIVE            '/dev/test/vo' [336.00 MiB] inherit
  ACTIVE            '/dev/rhel/swap' [<2.17 GiB] inherit
  ACTIVE            '/dev/rhel/root' [<26.83 GiB] inherit
$lsblk 
NAME          MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sr0            11:0    1 1024M  0 rom  
vda           252:0    0   30G  0 disk 
├─vda1        252:1    0    1G  0 part /boot
└─vda2        252:2    0   29G  0 part 
  ├─rhel-root 253:0    0 26.8G  0 lvm  /
  └─rhel-swap 253:1    0  2.2G  0 lvm  [SWAP]
vdb           252:16   0   10G  0 disk 
├─vdb1        252:17   0    2G  0 part 
│ └─test-vo   253:2    0  336M  0 lvm  /vo
└─vdb2        252:18   0    1G  0 part 
  └─myvg-mylv 253:3    0  800M  0 lvm  
vdc           252:32   0   10G  0 disk 
└─vdc1        252:33   0  500M  0 part /mnt
$

3. VDO volume

What is a VDO volume

Virtual data optimizer, which can deduplicate and filter data:

VDO（Virtual Data Optimize）It is RHEL8/Centos8a storage-related technology newly launched by Shanghai (the earliest test in the 7.5 beta version), and it is the technology of Permabit, which was acquired by Redhat.

VDOThe main function is to save disk space, for example, 1T disk can hold 1.5T data, thereby reducing the cost of data center.

How vdo is implemented : The key principles are mainly deduplication and compression. Deduplication is the copying of the same data from the hard disk, which used to take up multiple copies of space, but now only one copy of the space is needed. Similar to the way we upload a large software installation package in Baidu's network disk, it can achieve instant upload. In fact, it has been there before, so there is no need to upload it again, and there is no need to occupy Baidu's space. On the other hand, data compression, similar to the algorithm of compression software, can also save more disk space.

VDO是一个内核模块, the purpose is to reduce the space occupation of the disk and reduce the replication bandwidth through deduplication. VDO is based on the block device layer, that is, the mapper virtual device is mapped on the basis of the original device, and then it can be used directly. The realization of the function is mainly based on the following Technology:

Exclusion of zero blocks : During the initialization phase, the whole block of 0 will be recorded by the metadata. This can be explained by the example of mixing water and sand in a water glass, using filter paper (zero block exclusion), and sand (non-zero block) space) to filter out, and then the next stage of processing.

Deduplication : In the second stage, it is judged whether the input data is redundant data (before writing), and this part of the data is judged by the UDS kernel module (Universal Deduplication Service), which is judged as duplicate data The part of the data will not be written, and then the metadata will be updated to directly point to the original stored data block.

Compression : Once zero elimination and deduplication are completed, LZ4 compression processes each individual data block, and then the compressed data block is stored on the media as a fixed size 4KB data block, since a physical block can contain many Compressed blocks, this can also speed up read performance.

Basic operation of vdo:

• vdo create --name=VDO volume name --device=device path --vdoLogicalSize=logical size
• vdo list
• vdo status -n VDO volume name
• vdostats [–human-readable] [/dev/mapper/VDO volume name]
• vdo remove -n VDO volume name

Create VDO volume

$yum  -y  install  vdo     #装包
$systemctl restart vdo   #启动服务，并设置开机自启
# 找到vdc磁盘（此盘之前不能有配置）
$man  vdo   #搜索 /example ，在例子中查找相关配置命令
$vdo create --name=vdo0 --device=/dev/sdb1 --vdoLogicalSize=10T    #name是vdo卷的名称，device是使用哪个物理磁盘，最后是定义虚拟磁盘大小
 
$mkfs.xfs  -K   /dev/mapper/myvdo    #格式化 `-K可以加快速度`
#设置/etc/fstab文件，并且使用mount  -a检测
/dev/mapper/myvdo /vblock xfs _netdev 0 0      #	`_netdev的作用是等待网络相关程序启动后再执行挂载任务，通常网络启动之后vdo服务也就起来了，就可以挂载了`

Fourth, the exchange partition

What is a swap partition

相当于win的虚拟内存，在物理内存不足时借用硬盘空间,硬盘中预先划分一定的空间, and then temporarily store the temporarily unused data in the memory to the hard disk to free up physical memory space for more active program services to use. Its design purpose is to solve the problem of insufficient real physical memory.

However, since the swap partition reads and writes data through the hard disk device after all, the speed is definitely slower than the physical memory, so only when the real physical memory is exhausted will the resources of the swap partition be called.

Swap partition: virtual memory, generally set to 1~2 times of physical memory, <16G

format the swap partition

$swapon -s    #查询交换分区状态，通常有可能会存在个默认的交换分区，不能删除
# 先用fdisk /dev/vdb  再创建一个512M的分区比如创建了vdb3
$mkswap  /dev/vdb3    #格式化交换分区
$vim /etc/fstab   ##修改开机挂载磁盘的文件
/dev/vdb3   swap     swap    defaults        0 0
$swapon  -a  #测试上述文件是否修改正确
$swapon -s  #再次查看会多出swap分区

Enable/stop/view swap usage:

$swapon   设备路径
$swapoff   设备路径
$swapon  -s
$free

5. Actual combat

To resize the logical volume:

Resize the logical volume vo and its filesystem to 300MiB. Make sure that the file system contents remain unchanged.

$lvscan #找出要扩展的逻辑卷 
$lvextend -L 300MiB /dev/test/vo #扩展逻辑卷 
$blkid /dev/test/vo #检查文件系统格式 
$xfs_growfs 逻辑卷对应的挂载点 #适用于 XFS 文件系统 或者 
$resize2fs 逻辑卷对应的挂载点 #适用于 EXT2/3/4 文件系统

Add swap partition:

$fdisk /dev/vdb #修改磁盘 vdb
.. ..
Command (m for help): n #添加新分区
Partition number (2-128, default 2): #直接回车（默认）
First sector (4194304-20971486, default 4194304): #直接回车（默认）
Last sector, *sectors or +size{
    
    K,M,G,T,P} (4194304-20971486,default 20971486): +512M
Created a new partition 2 of type 'Linux filesystem' and of size 512 MiB.
Command (m for help): w #保存分区表，并退出
The partition table has been altered.
Syncing disks.
$partprobe /dev/vdb #刷新分区表
$mkswap /dev/vdb2 #格式化自建分区 vdb2
$vim /etc/fstab
/dev/vdb2 swap swap defaults 0 0
$swapon -a #启用 fstab 中的交换设备
$swapon -s #查看交换分区信息

Create volume group logical volumes:

Create a new logical volume with the following requirements:

• The name of the logical volume is mylv, it belongs to the volume group myvg, and the size is 50 expansion units
• Logical volumes in volume group myvg should have an extent size of 16MiB
• Format the logical volume mylv with the vfat filesystem
• This logical volume should be automatically mounted under the /mnt/mydata directory at system startup

$fdisk /dev/vdb $修改磁盘 vdb
.. ..
Command (m for help): n #添加新分区
Partition number (3-128, default 3): #直接回车（默认）
First sector (5242880-20971486, default 5242880): #直接回车（默认）
Last sector, *sectors or +size{
    
    K,M,G,T,P} (5242880-20971486,default 20971486): +1000M
Created a new partition 3 of type 'Linux filesystem' and of size 1000 MiB.
Command (m for help): w #保存分区表，并退出
The partition table has been altered.
Syncing disks.
$partprobe /dev/vdb #刷新分区表
$vgcreate -s 16MiB myvg /dev/vdb3 #建卷组（使用分区 vdb3）
$lvcreate -l 50 -n mylv myvg #建逻辑卷
$mkfs.vfat /dev/myvg/mylv #格式化
$mkdir /mnt/mydata #创建挂载点目录
$vim /etc/fstab #设置开机挂载
/dev/myvg/mylv /mnt/mydata vfat defaults 0 0
$mount -a #启用&测试开机挂载

Create a VDO volume using vdo

Create a new VDO volume with the following requirements:

• Use an unpartitioned disk (/dev/vdc)
• The name of this VDO volume is myvdo
• The logical size of this VDO volume is 50G
• This VDO volume is formatted with the xfs file system
• This VDO volume is automatically mounted to the /vblock directory at system startup

$yum install vdo #装包
$systemctl enable --now vdo #起服务
$vdo create --name=myvdo --device=/dev/vdc --vdoLogicalSize=50G
 #新建 VDO 卷
$mkfs.xfs -K /dev/mapper/myvdo #格式化
#或者 mkfs.ext4 -E nodiscard ...
$mkdir /vblock #创建挂载点目录
$vim /etc/fstab
/dev/mapper/myvdo /vblock xfs _netdev 0 0
$mount -a #启用&测试开机挂载