overlayfs
overlayfs attempts to provide a combined file system view on top of other file systems
Upper and Lower
overlayfs combines 2 file systems-Upper file system and Lower file system.
When the file with the same name exists in both Upper and Lower, the file in the Lower will be hidden, and if it is a directory, it will be merged and displayed.
The lower file system should be more accurately a directory tree, because these directory trees can belong to different file systems.
The lower file system can be any file system type supported by linux, and it can even be overlayfs.
Directory merge
For non-directory files, if both upper and lower exist, then the objects in the lower will be hidden.
If it is a directory, the directories in upper and lower will be merged.
We specify upperdir, lowerdir with the following command, they will be merged into merged directory.
mount -t overlay overlay -o lowerdir=/root/lowerdir/,upperdir=/root/overlayt/upper,workdir=/root/overlayt/work /root/overlayt/merged
The contents of the lowerdir directory are as follows:
[root@localhost overlayt]# ls -l /root/lowerdir/
Total consumption 4
-rw-r--r-- 1 root root 4 3月 15 18:35 1
-rw-r--r-- 1 root root 0 3月 15 18:35 2
-rw-r--r-- 1 root root 0 3月 15 18:35 3
drwxr-xr-x 2 root root 6 3月 15 18:35 4
[root@localhost overlayt]#
upper: our writable layer
lower: the bottom directory tree
merged: joint view
workdir: needs to be an empty directory and in the same file system as upper
When performing a lookup operation in the merged directory, lookup searches in each directory and caches the results in the dentry of the overlayfs file system.
If the same directory exists in upper and lower, it will be merged and displayed in merged
We create the low1 file in the dir1 directory in the lower
Create an up1 file in the dir1 directory of upper
Up1, low1 will appear in the dir1 directory in the merged.
Let's take a look at the handling of deleted files
whiteouts and opaque directory
In order to support rm, rmdir operations, and does not affect the contents of lowdir, overlayfs needs to record the information of the deleted file in the upper.
Whiteout is a character device file with a master and slave device number of 0/0. When there is a whiteout file in the merged directory, the file with the same name in the lower directory will be ignored. The whiteout file itself will also be hidden, and we can find it in the upper directory it.
readdir
When calling readdir in the merged directory, the files in the upper and lower directories will be read (first read the upper directory, then read the lower directory). This merged list will be cached in the file structure and kept until the file is closed. If the directory is opened and read by two different processes, then they have different caches. After seekdir sets the offset to 0, then calling readdir will invalidate the cache and rebuild it.
This means that changes in merged will not be reflected during the opening of dir, which is easy to ignore for many programs.
Non-catalog
For non-directory objects (files, symbolic links, special device files). When writing to a lowdir object, the copy_up operation is performed first. Copy_up is also required to create hard links, but not to soft links.
Copy_up may not be needed sometimes, for example, it is opened in read-write mode and is not actually modified.
The process of copy_up processing is roughly as follows:
- Create directory structure on demand
- Create file objects with the same metadata
- Copy file data
- Copy extended attributes
After copy_up is completed, overlayfs can simply provide access to objects on the upper file system.
Multiple lower layers
We can use ":" to specify multiple lower layers.
mount -t overlay overlay -olowerdir = / lower1: / lower2: / lower3 / merged
In the above example, we did not specify upper, workdir. This means that overlayfs is read-only.
Multiple lower layers are pushed into the stack from the right, and are reflected in the merge in the order in the stack. lower3 is at the bottom and lower1 is at the top.
Non-standard behavior
The copy_up operation creates a new file. The new file may be in a different file system than the old file, so st_dev and st_ino are new.
File locks on old files will not copy_up before copy_up
If a file with multiple hard links is copied_up, this link will be broken. Changes will not be propagated to the same hard-linked file.
Modify the underlying underlay file system
Offline modification, when overlay has no mount, allow modification of upper and lower directories
It is not allowed to modify the underlay when mounting overlay. If the underlay is modified, the behavior of the overlay is uncertain. Although it will not crash or deadlock.
