Ext4)

Ext

The full name is Linux extended file system, extfs, that is, Linux extended file system. Ext2 stands for the second generation file extension system. Ext3/Ext4 and so on. They are all upgraded versions of Ext2, just to quickly restore the file system and reduce consistency. The log function is added to the inspection time, so Ext2 is called an indexed file system , and Ext3/Ext4 is called a log file system .

Note: Linux supports many file systems, including Network File System (NFS), and Windows' Fat file system.

View file systems ls -l /lib/modules/$(uname -r)/kernel/fs

supported by Linux : View file systems supported by Linux (loaded into memory):cat /proc/filesystems

Core design

Data storage area

These elements are relatively stable and fixed after the disk is formatted.

The inode (index node)
records the file permissions, attributes, and blockthe number of the block where the data is located . Each file has one and only one, inodeand each inodehas its own number, which can be inodesimply understood as a document index .

Note: After the disk is formatted, the size and number of inodes have been fixed, and the size is 128Bytes (the new Ext4 and xfs are 258Bytes). When reading a file, first read the file attributes and permissions recorded in the inode, and then read the file content (block) only after the match is correct. In Linux systems, inodes are actually used to identify files instead of file names, similar to the design of user IDs and nicknames .

Inode table
stores all inode numbers of the file system
The file content stored in block (data block) , also called data block, each blockhas its own number, and the Ext2supported unit blockcapacity is only 1k, 2k, 4k .

Note: In order to facilitate inode recording, the block size has been fixed after the disk is formatted. Each block can only store the data of one file. If the file is too large, it will occupy multiple blocks; if the file is too small, the remaining space of the block cannot be used, which will result in a waste of disk space . Therefore, after the disk is partitioned, the file Before formatting the system, please think carefully about the expected usage of the file system .

View the status of a file or file system

stat [options] [filename]

View the inode usage of each file system of the system

df -i

Intermediary data (metadata)

These elements are designed to maintain the state of the file system. When adding, editing, or deleting documents, you need to change the state information.

The superblock (super block)
records the overall information of the file system, including inode/blockthe total amount, usage, remaining amount, size, and the format and related information of the file system.

Note: The basic information of the entire file system is all recorded in the superblock, and its size is generally 1024Bytes. If it dies, it will take a lot of time to remedy it! ! !

Block group (block group)
Imagine if our disk capacity is as high as hundreds of gigabytes. After we format, the inode and block will be very large. In order to facilitate management, the Ext file system introduces block group when format The concept of group (block group), each block group maintains an independent inode/block/superblock, with a fixed number of blocks, which is divided into a group of most basic sub-file systems.

Note: Superblock is too important for the file system, but there is only one superblock in the file system, so in addition to the first block group containing superblock, subsequent block groups may contain backup superblocks, the purpose is to avoid superblock single point cannot be rescued The problem.

Block bitmap (block comparison table)
A block can only be used by one file. When we add a file, we must use the new block to record the file data. So how do you quickly know which blocks are new? Which blocks are already used? The block bitmap is designed in this way, recording all used and unused block numbers. Similarly, when we delete a file, first find the corresponding block number from the block bitmap, then update the flag as unused, and finally release the block.
The inode bitmap (inode comparison table) has the
same design concept as the block bitmap, except that it records the used and unused inode numbers, which will not be described here.
The group descriptor
describes the start and end block numbers of each block (block group), and explains which block numbers each block (inodemap, blockmap, inode table) is between.

List all formatted devices in the current system: blkid

select a formatted device to view the detailed information of the file system: dumpe2fs /dev/vda1

Note: The Magic signature above is 0xEF53, indicating that our disk partition is a standard ext2 and ext3 file system. Similar to Magic through the beginning of the file, the file type can be judged the same.

Example description

1. The role of inode

When a user searches or accesses a file, the UNIX system uses the inode table to find the correct inode number. After finding the inode number, related commands can access the inode and make appropriate changes.

Example

For example, use vito edit a file. When you type vi<filename>, you will be allowed to open the inode only after the inode number is found in the inode table. During the editing session of vi, some attributes in the inode were changed. When you finish the operation and type :wq, the inode will be closed and released. In this way, if two users try to edit the same file, the inode has already been assigned to another user ID (UID) during the first editing session, so the second editing task must wait until the inode Until released.

Note: You can refer to Baidu Encyclopedia.

2. The importance of block

Through the above analysis, we know that block is the atomic unit of file data storage, and each block can only store the data of one file. When formatting a file system, if you choose improperly, a lot of disk space will be wasted.

Example

If the block selected for the file system is 4k and stores 10,000 small files, each 500 bytes, how much disk space is wasted at this time?
Disk capacity wasted per file = 4096-500 = 3596bytes, disk capacity wasted by 10,000 files = 10000 * 3596 ~=34M, actual file capacity = 10000 * 500 ~=4.7M, there is no harm without comparison, actual storage If the capacity is less than 5M, 34M was wasted, with a waste rate of 680%, and the more files the more serious the waste.

Remarks: From a theoretical analysis, only when the actual file capacity is equal to the system's minimum storage unit capacity, the disk will not be wasted, but this is an ideal situation, then we can choose the smallest block, and there is nothing wrong. However, at this time, a new problem has arisen. Large files will occupy too many blocks, causing too many block numbers to be recorded by the inode, and the read and write performance of the file system will decrease. Therefore, we must have a degree in everything. With this degree, the overall performance and utilization of the file system can be improved.

3. The relationship between inode and block and file size

The data is actually stored in blocks. In order to read files quickly, each file corresponds to an inode index file and records all block numbers. However, the size of the inode is only 128bytes or 256bytes (ext4). If a file is too large, the number of blocks It is likely to exceed the recordable number of inodes. For this reason, the area where the inode records the block number is designed as 12 direct, one indirect, one double indirect, and one triple indirect record areas.

Remarks: The so-called indirection is to use a block as the block number recording area. Only the last indirection is actually used to record the block number, and the other indirect layers are only quoted in turn.

Calculate the maximum capacity of a single file

Each block number is a number and needs to occupy 4 bytes.

Assuming that the unit capacity of a block is 1K, the block number that each block can record is 1k/4=256 .
- 12 direct capacity = 12 * 1k = 12k
- Single indirect capacity = 256 * 1k = 256k
- Double indirect capacity = 256 * 256 * 1k = 65536k
- Three indirect capacity = 256 * 256 * 256 * 1k = 16777216k
- Maximum total amount of a single file = 12 direct capacity + single indirect capacity + double indirect capacity + triple indirect capacity = (12 + 256 + 65536 + 16777216) / (1024 * 1024) = 16.06G
Assuming that the unit capacity of a block is 2K, the block number that each block can record is 2k/4=512 .
- 12 direct capacity = 12 * 2k = 24k
- Single indirect capacity = 512 * 2k = 1024k
- Double indirect capacity = 512 * 512 * 2k = 524288k
- Three indirect capacity = 512 * 512 * 512 * 2k = 268435456k
- Maximum total amount of a single file = 12 direct capacity + single indirect capacity + double indirect capacity + triple indirect capacity = (24 + 1024 + 524288 + 268435456) / (1024 * 1024) = 256.50G
Assuming that the unit capacity of a block is 4K, the block number that each block can record is 4k/4=1024 .
- Similarly, the maximum total volume of a single file = 12 direct capacity + single indirect capacity + double indirect capacity + three indirect capacity = 4.00T

Linux standard file system restriction table

Note: When the unit capacity of the block is 4K, due to the limitation of the file system (2T), it does not agree with the calculated result.

Check the capacity of disks and documents

1. View the overall disk capacity of the file system

df  [-ahikHTm] [目录或文件名]

2. View directory and file capacity

du [options] []

View the capacity of all directories under the directory geekbuying

du -sm geekbuying/*

Statistics the current directory capacity

du -sm 单位M

to sum up

The Ext family is the most widely supported and complete file system in Linux. When we format the disk, we have already planned all the inode/block/metadate data for us, so that the system can be used directly without any dynamics. This is also its best feature, but this is also its most significant shortcoming. The larger the disk capacity, the slower the formatting. Centos7.x has selected xfs as the default file system. Xfs is a kind of suitable for large-capacity disks and A file system that handles huge files.