For system administrators, it is very necessary to understand some limitations of the system, so that they can configure and adjust the necessary parameters according to needs, and then achieve better performance. For system designers and even programmers, understanding the system Some restrictions will also help to design a more reasonable storage structure.
1. Limitation on the number of directories In
RedHat Enterprise Linux AS 4.0 Update 3
, the number of first-level subdirectories is limited to 31998 under the same path. If the number of directories generated by your application may exceed this number, you must pay attention to the directory classification. For example, if the directory name is a number, the integer value after dividing the number by 10000 can be used as the parent directory name (/data/1/13892), if the directory name is a letter, you can use the first few characters as The parent directory (/data/ab/abcdefg) and so on.
You can get the total number of subdirectories under a directory through the following command line:
$ ls -F | grep "/" | wc
Solution:
1) Create a subdirectory
2) Linux stipulates for the search efficiency of cpu, if you want Changing the number probably requires recompiling the kernel.
Second, the limit on the number of files.
Each file corresponds to an inode. If the inode is gone, no more files can be written even if there is space.
Through the following command line, you can view the inodes information of a partition:
[root@boss tmp]# tune2fs -l /dev/sda5 | grep "inodes"
Free inodes: 1247005
[root@boss tmp]#
Two solutions:
1) Re-mkfs, when mkfs adjust the number of inodes more (according to the total number of files in your fs), and adjust the block size to be smaller (according to the average size of each file) 2) Use the loopback file system
temporarily Solution
Create a large file in /usr (or elsewhere), then make a loopback file system, move the original file to this file system, and mount it to a suitable location under /usr. This can greatly reduce the number of files in your /usr. But system performance will suffer a bit.
3 Limits on the number of files in a single directory
I thought there was a limit on the number of files in a single directory under Linux, but I didn't find a clear statement after a Google search, but there is a limit of 65535 under Freebsd, which can be changed by recompiling the kernel. (You can look at /usr/include/sys/syslimits.h)
However, it is not recommended to put too many files in one directory, which will affect system performance.
Supplement: The ulimit -a command can view all limits and can temporarily change the limits.
The number of first-level subdirectories in the ext3 file system is 31998 (units) by default, 32000 to be precise.
Linux stipulates for the search efficiency of cpu, if you want to change the number limit, you need to recompile the kernel. I saw this in the kernel code:
include/linux/ext2_fs.h:#define EXT2_LINK_MAX 32000
include/linux/ext3_fs.h:#define EXT3_LINK_MAX 32000
Why do you say 31998?
This is because when mkdir creates a directory, two subdirectories will be created under the directory by default, one is the . directory (representing the current directory), and the other is the .. directory (representing the parent directory). These two subdirectories cannot be deleted, " rm . " will get a prompt of "rm: cannot remove `.' or `..'". So 32000-2=31998.
Alternatively, you can try the following script.
#!/bin/bash
mkdir tmp
cd tmp
i=1
while [ $i -lt 35000 ]
do
mkdir $i
if [ $? -ne 0 ]; then
echo "cannot make dir $i"
exit
fi
((i++))
doneRun
this script, you will end up with "mkdir: cannot create directory `31999': Too many links" error message.
It is not recommended to have too many files or directories in one directory
This degrades the performance of the file system for finding files or directories. It suddenly occurred to me that Alibaba's image server divides the storage of images into subdirectories at different levels according to the year and month instead of in one directory. One of the reasons is also due to the consideration of the performance of the Linux operating system.
