The difference between Fsimage and Edits in HDFS file system

Correlation probability

Fsimage image file: a persistent checkpoint of metadata, including metadata information of all directories and files in the Hadoop file system, but not the location of file blocks. The file block location information is only stored in the memory. It is obtained by the NameNode asking the DataNode when the DataNode joins the cluster, and it is updated intermittently.

Edits edit log : It stores all the change operations (file creation, deletion or modification) of the Hadoop file system. The change operations performed by the file system client will first be recorded in the Edits file.

Same point

Both the Fsimage and Edits files are serialized. When the NameNode starts, it will load the contents of the Fsimage file into the memory, and then perform various operations in the Edits file to synchronize the metadata in the memory with the actual , The metadata stored in the memory supports the client read operation, which is also complete metadata

difference

When the client adds or modifies a file in HDFS, the operation record is first recorded in the Edits log file. When the client operation is successful, the corresponding metadata will be updated to the memory data. Because Fsimage files are generally very large (GB level is very common), if all update operations are added to the Fsimage file, this will cause the system to run very slowly.

Guess you like

Origin blog.csdn.net/abc5254065/article/details/112982676