My needs:

Preparing for the Red Hat exam, this lecture was very confused.

I did this:

I consulted the company boss and found some information in the book. Here are the notes to be memorized. Please point out the shortcomings. Come on life. 2021.01.08

1. Mind mapping to sort out the relationship

2. Linux file storage (LVM is not used)

If we normally get a disk, we will use lsblk to view the partition, and then use fdisk for partitioning. After partitioning, we will use the mkfs-related specified file system for formatting, and finally mount it through mount, which is the physical disk area in the figure above. .

Identify the disk --> Partition --> Format --> Mount --> Access the mount point: lsblk --> fdisk -l --> fdisk(MSDOS partition table)/gdisk(GPT partition table>2.2TB disk )/parted --> mkfs related --> mount

Creation process:

lsblk    //查看分区状态
fdisk /dev/vdc     //使用fdisk分区工具，对/dev/vdc硬盘进行分区
p 查询分区状态 			
q 不保存退出
n 新建分区
Select (default p):      //回车，默认是主分区
Partition number (1-4, default 1):     //第一个分区默认序号是1
First sector ：    //起始扇区，直接回车
Last sector       +500M      //创建500M分区
w    //保存退出
mkfs.xfs /dev/vdc1     //使用xfs文件系统为vdc1这个分区格式化
mount   /dev/vdc1   /mnt    //将新分区挂载到/mnt目录
df -h   
umount   /dev/vdc1   //卸载分区

//刷新硬盘分区表：
partprobe /dev/vdb 或者 partx -a /dev/vdb
reboot

Third, use LVM file storage

The Linux logical volume manager (LVM) allows you to easily manage disk space without rebuilding the entire file system. Using standard partitions to create a file system on the hard disk, adding extra space to the existing file system is a painful experience. You can only adjust the partition size within the available space of the same physical hard disk.

A volume group (VG) can be formed by gathering multiple physical volumes (physical partitions) together through LVM . The logical volume management system regards the volume group as a physical hard disk , but in fact the volume group may be composed of multiple physical partitions distributed on multiple physical hard disks . Volume groups provide a platform for creating logical partitions, and these logical partitions contain file systems.

The last layer in the entire structure is the logical volume (LV) . Logical volumes provide Linux with a partition environment for creating file systems , which are similar to the physical hard disk partitions in Linux that we have been discussing so far . Linux systems treat logical volumes as physical partitions. You can use any standard Linux file system to format the logical volume, and then add it to a mount point in the Linux virtual directory.

LVM逻辑卷The idea of management mechanism —— 化零（物理卷PV）为整（卷组VG）、动态扩容伸缩，按需（逻辑卷LV）分配。

Distributed partitions (PV physical device (physical volume)) ===》Reorganized large volume group (VG virtual disk) ===》 Obtain space as needed (virtual partition LV)

The connection between physical volumes, logical volumes, volume groups, and snapshot volumes

Physical Volume (PV) : Refers to the hard disk partition, it can also be the entire hard disk or the created soft RAID, which is the basic storage device of LVM.
Volume Group (Volume Group, VG) : A storage pool composed of one or more physical volumes. One or more logical volumes can be created on the volume group.
Logical Volume (LV) : Similar to the hard disk partition in a non-LVM system, it is built on a volume group and is a standard block device. A file system can be built on the logical volume.

We can make this assumption to understand the relationship between the above three: If PV is compared to a plate of the earth, VG is an earth, because the earth is composed of multiple plates, then divide a region on the earth and mark it as Asia , Then Asia is equivalent to an LV.

Interrelationship: When creating a volume group, you must reserve space for logical volume snapshots , and then the snapshot accesses another entry of the logical volume. As long as the physical volume is added to the volume group, the physical space provided by the physical volume is reserved in advance. It is divided into blocks, and this block is called PE (Physical Extend) before being formatted . It is a small box of logical storage. The size of the volume group is composed of multiple PEs , while the size of the logical volume The size is to put the PE in the volume group into the logical volume. At this time, the PE is no longer called PE, but called LE (Logical Extend) [logical extent] . In fact, the LE in the logical volume is also called PE, which is just a station. The angle is different .

If a physical volume is damaged , the LE stored in the logical volume will also be damaged . If you want the data not to be damaged, you can make the PE in the physical volume a mirror

LVM in Linux

1. Snapshot The original Linux LVM allowed you to copy the logical volume to another device while it was online .

LVM1 only allows you to create read-only snapshots. Once the snapshot is created, no more things can be written.
LVM2 allows you to create read-write snapshots of online logical volumes. With a readable and writable snapshot, you can delete the original logical volume, and then mount the snapshot as a replacement. This feature is very useful for fast failover or program trials involving modified data (if it fails, the modified data needs to be restored).

Snapshots are very useful when backing up important data that cannot be locked due to high reliability requirements. Traditional backup methods usually lock files when copying files to backup media. Snapshots allow you to keep running critical tasks while copying.

2. striping , LVM2 supplied Another striking feature is the striping (striping). Ma Yushu

Striping can create logical volumes across multiple physical hard drives. When Linux LVM writes a file to a logical volume, the data blocks in the file will be scattered across multiple hard disks. Each subsequent data block will be written to the next hard disk.
Striping helps to improve the performance of the hard disk, because Linux can write multiple data blocks of a file to multiple hard disks at the same time, without waiting for a single hard disk to move the read and write heads to multiple different locations. This improvement also applies to reading files that are accessed sequentially, because LVM can read data from multiple hard drives at the same time. (Multiplexing).

LVM striping is different from RAID striping. LVM striping does not provide verification information to create a fault-tolerant environment. In fact, LVM striping increases the probability of file loss due to hard disk failure. A single hard disk failure may cause multiple logical volumes to become inaccessible.

3. Mirror LVM mirroring. A mirror is a complete copy of a logical volume that is updated in real time. When you create a mirrored logical volume, LVM will synchronize the original logical volume to the mirrored copy. Depending on the size of the original logical volume, this may take some time to complete. Once the original synchronization is completed, LVM will perform two writes for each write operation of the file system-once to the primary logical volume and once to the mirror copy. It is conceivable that this process will reduce the write performance of the system. Even if the original logical volume is damaged for some reason, you already have a complete and up-to-date copy on hand!

LVM management tools

Physical volume operations (not commonly used): pvscan, pvdisplay, pvcreate
Volume group operations: vgscan (scan), vgdisplay (display), vgcreate, vgremove, vgextend (extend)
Logical volume operations: lvscan, lvdisplay, lvcreate, lvremove, lvextend

PE: When allocating logical volume space, the smallest unit, the default is 4M

Create a volume group: Logical volumes are divided into integers to use disk space more freely 创建逻辑卷之前先要创建卷组, and then from卷组中划分空间给逻辑卷

   vgcreate  [-s  PE大小]  卷组名  分区.. ..

vgcreate /dev/myvg /dev/vdb2     //正常创建卷组，默认扩展单元是4M（不满足目前题意，目前要求是16M）
vgdisplay  /dev/myvg   //可以在PE一栏中看到扩张单元的大小
vgremove /dev/myvg   //删除卷组myvg
vgcreate -s 16MiB /dev/myvg /dev/vdb2    //创建扩展单元为16MiB的卷组（目前练习题要求），之后可以用vgdisplay查看PE的大小

Create a logical volume:

   lvcreate  -L  大小  -n  名称   卷组名
   lvcreate  -l  PE个数  -n  名称   卷组名

lvcreate -n mylv -L 800M /dev/myvg    //正常创建
lvcreate -n mylv -l 50 /dev/myvg    //按照扩展单元的数量创建（目前练习题要求），之后可以用vgs查看
vgs     // 查看邏輯卷

Extend the logical volume:

   lvcreate  -L  大小  -n  名称   卷组名
   lvcreate  -l  PE个数  -n  名称   卷组名

lvextend -L 300M /dev/test/vo    //将名字叫vo的逻辑卷扩容到300M
blkid /dev/test/vo    //查看vo逻辑卷的文件系统
lsblk    //再查看该逻辑卷的挂载点
xfs_growfs /vo    //刷新大小，如果是xfs的文件系统
resize2fs  /vo     //刷新大小，如果是ext的文件系统

Extend the volume group:

   vgextend  卷组名  分区.. ..

Notify the kernel of the new file system size:

   resize2fs  /dev/卷组名/逻辑卷名 //适用于EXT2/3/4文件系统（blkid检查）
   xfs_growfs  逻辑卷名的挂载点//适用于XFS文件系统（blkid检查）

Device location of the logical volume:

/dev/卷组名/逻辑卷名
或者
/dev/mapper/卷组名-逻辑卷名

Scan disk LVM information

[root@blue ~]# vgscan 
  Reading all physical volumes.  This may take a while...
  Found volume group "myvg" using metadata type lvm2
  Found volume group "test" using metadata type lvm2
  Found volume group "rhel" using metadata type lvm2
[root@blue ~]# pvscan
  PV /dev/vdb2   VG myvg            lvm2 [1008.00 MiB / 208.00 MiB free]
  PV /dev/vdb1   VG test            lvm2 [<2.00 GiB / <1.67 GiB free]
  PV /dev/vda2   VG rhel            lvm2 [<29.00 GiB / 0    free]
  Total: 3 [<31.98 GiB] / in use: 3 [<31.98 GiB] / in no VG: 0 [0   ]
[root@blue ~]# lvscan 
  ACTIVE            '/dev/myvg/mylv' [800.00 MiB] inherit
  ACTIVE            '/dev/test/vo' [336.00 MiB] inherit
  ACTIVE            '/dev/rhel/swap' [<2.17 GiB] inherit
  ACTIVE            '/dev/rhel/root' [<26.83 GiB] inherit
[root@blue ~]# lsblk 
NAME          MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sr0            11:0    1 1024M  0 rom  
vda           252:0    0   30G  0 disk 
├─vda1        252:1    0    1G  0 part /boot
└─vda2        252:2    0   29G  0 part 
  ├─rhel-root 253:0    0 26.8G  0 lvm  /
  └─rhel-swap 253:1    0  2.2G  0 lvm  [SWAP]
vdb           252:16   0   10G  0 disk 
├─vdb1        252:17   0    2G  0 part 
│ └─test-vo   253:2    0  336M  0 lvm  /vo
└─vdb2        252:18   0    1G  0 part 
  └─myvg-mylv 253:3    0  800M  0 lvm  
vdc           252:32   0   10G  0 disk 
└─vdc1        252:33   0  500M  0 part /mnt
[root@blue ~]#

Fourth, other logical volumes, partition concepts

VDO volume

The virtual data optimizer can deduplicate and filter data:

VDO (Virtual Data Optimize) is a storage-related technology newly launched on RHEL8/Centos8 (tested in the 7.5 beta version at the earliest), and is a technology of Permabit acquired by Redhat.
The main function of VDO is to save disk space. For example, a 1T disk can hold 1.5T of data, thereby reducing the cost of the data center.

How does vdo realize: The key principle is mainly deduplication and compression. Deduplication means copying the same data from the hard disk. It used to take up multiple copies of space, but now it only needs one copy of space. Similar to how we upload a large software installation package in Baidu's network disk, it can be uploaded in seconds. In fact, it has been there before, so there is no need to upload it again, and there is no need to occupy Baidu's space. On the other hand, data compression, similar to the algorithm of compression software, can also save disk space.

VDO is a kernel module. Its purpose is to reduce disk space occupation and copy bandwidth through deduplication. VDO is based on the block device layer, which is to map the mapper virtual device on the basis of the original device, and then use it directly. The realization of is mainly based on the following technologies.

1) Elimination of zero blocks: In the initialization phase, the entire block of zero will be recorded by metadata. This can be explained by the example of mixing water and sand in the water cup. Use filter paper (zero block elimination) to remove the sand ( Non-zero space) to filter out, and then the next stage of processing.
2) Deduplication: In the second stage, the input data will be judged whether it is redundant data (judging before writing). This part of the data is judged by the UDS kernel module (Universal Deduplication Service) and is judged as The part of the repeated data will not be written, and then the metadata will be updated to directly point to the original data block that has been stored.
3) Compression: Once the zero suppression and deduplication are completed, LZ4 compression will process each individual data block, and then the compressed data block will be stored on the medium as a fixed size 4KB data block, because a physical block can contain A lot of compressed blocks, this can also speed up the read performance.

Basic operations of vdo:

vdo create --name=VDO volume name--device=device path--vdoLogicalSize=logical size
vdo list
vdo status -n VDO volume name
vdostats [--human-readable] [/dev/mapper/VDO volume name]
vdo remove -n VDO volume name

yum  -y  install  vdo     //装包
systemctl restart vdo   //启动服务，并设置开机自启
找到vdc磁盘（此盘之前不能有配置）
man  vdo   //搜索 /example ，在例子中查找相关配置命令
vdo create --name=vdo0 --device=/dev/sdb1 --vdoLogicalSize=10T    //然后按要求稍作修改，name是vdo卷的名称，device是使用哪个物理磁盘，最后是定义虚拟磁盘大小
#格式化 
mkfs.xfs  -K   /dev/mapper/myvdo      // `-K可以加快速度`
#设置/etc/fstab文件，并且使用mount  -a检测
/dev/mapper/myvdo /vblock xfs _netdev 0 0      //	`_netdev的作用是等待网络相关程序启动后再执行挂载任务，通常网络启动之后vdo服务也就起来了，就可以挂载了`

Swap partition

It is equivalent to the virtual memory of win. When the physical memory is insufficient, the hard disk space is borrowed. A certain amount of space is pre-divided in the hard disk, and then temporarily unused data in the memory will be temporarily stored in the hard disk to free up the physical memory space for more active The technology used by the program service is designed to solve the problem of insufficient real physical memory.
But since the swap partition reads and writes data through the hard disk device after all, the speed is definitely slower than the physical memory, so the swap partition resources will only be called when the real physical memory is exhausted.

Swap partition: virtual memory, generally set to 1~2 times of physical memory, <16G

Format swap partition

swapon -s    //查询交换分区状态，通常有可能会存在个默认的交换分区，不能删除
先用fdisk /dev/vdb  //再创建一个512M的分区比如创建了vdb3
mkswap  /dev/vdb3    //格式化交换分区
vim /etc/fstab   //修改开机挂载磁盘的文件
/dev/vdb3   swap     swap    defaults        0 0
swapon  -a  //测试上述文件是否修改正确
swapon -s  //再次查看会多出swap分区

启用/停止控制：

 swapon   设备路径
 swapoff   设备路径

View the usage of the swap partition:

 swapon  -s
 free

Five, the mounting method of the file system

1) Manually mount:

mount 设备路径挂载点目录

2) Automatically mount at boot:

修改 /etc/fstab 配置文件
设备路径挂载点目录文件系统类型 defaults,_netdev 0 0

3) Semi-automatic mounting (/etc/fstab has been configured):

mount device path
mount mount point directory

4) Trigger mounting (automatic mounting on demand):`autofs服务`

Software package/system service:autofs nfs-utils
Main placement:/etc/auto.master
1. 目标文件夹监控策略配置文件
Monitoring strategy configuration:/etc/*
1. Subdirectory name-mount option: device path
2. Subdirectory name-mount option NFS server address: absolute path of the shared directory

By deploying the autofs service, the following tasks can be automatically completed on-demand instead of the administrator:

1) When a user needs to use the ldapuser0 subdirectory under the /rhome/ directory, create this directory in advance, and mount: /rhome/ldapuser0 on the server to the local /rhome/ldapuser0
2) When the user no longer uses the ldapuser0 subdirectory under the /rhome/ directory, the resources mounted on this directory are automatically uninstalled, and this directory is deleted

5）Demo

showmount  -e server1
yum -y install autofs nfs-utils   安装所需软件包
vim /etc/auto.master      修改主配置文件
/rhome /etc/rhome.rule   在第一行添加，/rhome是在本地什么位置作为远程资源的挂载目录，/etc/rhome.rule是远程资源配置文件所在路径，可以自定义
vim /etc/rhome.rule   修改上一步定义的远程资源配置文件，添加以下内容：
ldapuser0 -fstype=nfs 172.25.0.254:/rhome/ldapuser0 定义账户、文件系统类型、远程资源路径
systemctl restart autofs   重启服务
ls /rhome   查看原本没有的/rhome目录，发现已经自动生成
ls rhome/ldapuser0   再查看里面的内容，报权限错误 ，正常，已经激活自动挂载
ls /rhome  再次查看目录，已经看到远程资源，成功

Theoretical knowledge of logical volumes/physical partitions in Linux

1. Mind mapping to sort out the relationship

2. Linux file storage (LVM is not used)

Creation process:

Third, use LVM file storage

The connection between physical volumes, logical volumes, volume groups, and snapshot volumes

LVM in Linux

LVM management tools

Fourth, other logical volumes, partition concepts

VDO volume

Basic operations of vdo:

Swap partition

Five, the mounting method of the file system

1) Manually mount:

2) Automatically mount at boot:

3) Semi-automatic mounting (/etc/fstab has been configured):

4) Trigger mounting (automatic mounting on demand):`autofs服务`

5）Demo

Guess you like

Theoretical knowledge of logical volumes/physical partitions in Linux

1. Mind mapping to sort out the relationship

2. Linux file storage (LVM is not used)

Creation process:

Third, use LVM file storage

The connection between physical volumes, logical volumes, volume groups, and snapshot volumes

LVM in Linux

LVM management tools

Fourth, other logical volumes, partition concepts

VDO volume

Basic operations of vdo:

Swap partition

Five, the mounting method of the file system

1) Manually mount:

2) Automatically mount at boot:

3) Semi-automatic mounting (/etc/fstab has been configured):

4) Trigger mounting (automatic mounting on demand):autofs服务

5）Demo

Guess you like

4) Trigger mounting (automatic mounting on demand):`autofs服务`