Algorithms for snooping space and file size from ext3 and ext4 filesystems
To learn the operating system, you have to study the disk and the disk file system. The disk is the underlying physical device, and the file system is the upper-level tool for managing the disk. The file system plans the format of the disk to store data and determines how much disk an operating system can support. space, how much data space each partition can support, and how much each file can support. Usually for system administrators, the most important thing to know is the maximum disk space, the maximum partition space and the maximum file size. This topic only discusses how these three sizes are calculated, not rote memorization. Knowing the principle, no matter what file system you encounter in the future, there will be rules to follow, at least knowing the data you need to calculate the size of these three requirements. In addition, the composition of the disk and the layout of the file system are not discussed here, but only the calculation method, and the relevant knowledge points that may be involved, please check the relevant information.
Many people may memorize several values, but do not know why this value is, here is the analysis:
1. Hard and fast rules
ext3：
1). The ext3 file system uses a 32-bit block address index space;
2) In the inode entry, referencing a block space symbol requires a size of 4 bytes;
3) For an inode, 12 direct pointer indexes, one indirect pointer index, one double indirect pointer index, and one triple indirect pointer index are designed
Note:
a. These regulations are determined by the program code of the file system itself, that is to say, it is designed at the time of development, there is no why, only what; the same is true for other file systems;
b. The so-called double indirect pointer index and triple indirect pointer index refer to two-level structure and three-level structure, which are equivalent to the root file system directory tree in linux;
Attached is the internal structure diagram of the inode in the ext3 file system:
ext4:
1). The ext4 file system uses a 48-bit block address index space;
2). In the inode entry, instead of using the pointer index to map with the block, the extent is used instead of the pointer; the 15 previously in ext3
The pointer is replaced with 5 extents, one extent occupies 3byte space; an extent describes a group of consecutive blocks, when the extent is not enough, the extent can still be used
Indexes using indirect pointers, but no limit.
Note : For the algorithm for calculating the size of a single file by extent, I have not yet figured it out, and I have not understood the source code. If you understand the message.
2. Algorithm
Well, knowing the above source code design rules, the following can be calculated.
ext3:
1). Maximum supported file size
First of all, you must know that in the linux file system, the size of a block can be 1k, 2k, 4k, and when the block size is 4k, it is the largest. In the Linux system, each file must use an inode number, so to calculate the maximum space supported by a single file, you only need to know how many blocks can be referenced in the inode, and the value calculated when the block takes the maximum value of 4k It is the maximum space that a single file can support.
As can be seen from the hard and fast rules above:
a. An inode supports 12 direct pointers, so it represents 12 blocks
b. An inode supports an indirect pointer, that is, a pointer points to a block block, and the block block is used as a direct pointer to the final block. Here , a block is 4K, and a block takes up 4 bytes of space, so a block The indirect pointer can point to a 4K/4byte block, that is, 2^10
c. The double indirect pointer eventually points to: 2^10*2^10=2^20 blocks
d. Three indirect pointers are finally executed: 2^10*2^10*2^10=2^30 blocks
So the final number of blocks is: 12+2^10+2^20+2^30
Then the final size is: (12+2^10+2^20+2^30)x4KByte
Then the size converted into TB is: (12+2^10+2^20+2^30)x4KByte/1024/1024/1024=4TB
At this point, it can be seen that the ext3 file system supports the largest single file size when the block is 4K, and the maximum is 4TB.
Note : Of course, this is a theoretical value. It should be known in the industry that there will always be a little deviation between the actual value and the theoretical value, but it is harmless.
2)最大分区大小(即文件系统大小)
在操作系统中，文件系统都是针对分区而言的，一个磁盘必须先分区才能格式化文件系统(即使你将磁盘所有容量划分一个分区).格式文件系统后才能挂载使用，此时就必须知道一个文件系统到底支持多大的分区大小。
在ext3文件系统中，采用32bit的块索引空间，且其采用int的无符号整型，因此一个分区的最大空间为：
2^32*4KByte=16TB
由此，得知在ext3文件系统中，当block为4K时，一个分区的空间将最大，且最大空间为16TB
3)最大磁盘容量
有时候当我们需要更大的磁盘容量空间的时候，会增加磁盘来达到需求。但是最后分区使用时会发现提示不支持这么大的空间。这是为什么？
对于整块磁盘而言，因为存在MBR区域，而MBR中有64byte的空间只能表示4个分区，每个分区16btye来表示，因此对于ext3文件系统支持的最大分区为16TB，那最大系统磁盘容量为：16TB*4=64TB
注：这里有人要问几个问题了
a.有人问：你这里说4个分区，不是有逻辑分区吗，使用逻辑分区划分磁盘可以划分很多磁盘分区。那不是应该无限制吗?
answer：逻辑分区是扩展了磁盘分区的个数，但是逻辑分区也是从4个主分区中的一个主分区扩展出来的，因此所有逻辑分区的空间也不过是最后一个最大主分区的空间扩展出来的。
b.有人问：不对，在linux系统中，新磁盘可以作为分区后挂载到系统中的某个目录下来使用。
answer: Although the new disk can be mounted to a directory in the system for use, the new disk is also under the ext3 file system, and the maximum size of the new disk itself is 64TB. Therefore, this is based on the maximum disk capacity that can be supported when the block size is 4K under the ext3 file system. If this size is exceeded, the partition table cannot be recognized. Don't understand it as the amount of space that the operating system can use.
ext4:
1) Maximum space for a single file
Since ext4 no longer uses the form of pointers for block mapping by default, but uses extent to manage the number of blocks that can be described, I have not yet understood what the mechanism of extent is, so I do not do the calculation. But the consensus is that a single file can be as large as 16TB, which is equivalent to the maximum partition size of ext3.
2) Maximum partition size (ie file system size)
ext4 uses a 48bit block address index space, so when the block is 4k, the size is:
2^48*4KByte=1EB=1024PB=1024*1024TB
This space can be said to be sufficient in the current environment.
3) Maximum disk capacity
According to the analysis conclusion of ext3, we can see: 1EB*4=4EB
OK, that's all I have said, I don't know if you understand it. Anyway, I get it. (^_^)
Reprinted to http://blog.51cto.com/mingyang/1580314