nameNode
When nameNode starts, it first loads the image file (fsimage) into storage, and executes various operations in edits. Once the file system metadata image is successfully established in the memory, a new fsimage file and an empty edits (edits) are created. Log), at this time nameNode starts to monitor dataNode requests. At this moment, nameNode is running in safe mode, that is, the file system of nameNode is read-only for the client.
dataNode
When the dataNode starts, each dataNode scans the local disk and reports the block information stored on the dataNode to the nameNode, and the nameNode receives the block information and saves it in the memory.
After the dataNode is started, it will register with the nameNode and report all block information to the nameNode periodically (1 hour).
1. Keep in touch by sending a heartbeat to the nameNdoe (every 3 seconds).
2. If the heartbeat of DatraNode is not received in 10 minutes, it is considered This dataNode has gone down, and the nameNode starts the process to restore the data blocks on it to other dataNodes.
3. The dataNode verifies whether its checkSum value is the same as the checkSum value when the file is created three weeks after its file is created.
[Note]
1. The nameNode does not respond to the dataNode request. In this communication process, the client/server architecture is strictly adhered to, and
dataNodes can communicate and perform data block replication tasks. At the same time, when the client performs a write operation, the dataNodes need to cooperate with each other to establish a data pipeline.
2. The location of the data block in the system is not maintained by the nameNode, but stored in the dataNode in the form of a quick list. During the normal operation of the system, the nameNode will keep the mapping information () of all block locations in the memory. In the safe mode, each dataNode will send the latest block list information to the nameNode. Meta-information content stored by nameNode: file directory tree structure, file-blockid mapping relationship, blockid-datanode mapping relationship
Reference: https://blog.csdn.net/lb812913059/article/details/79717525?utm_medium=distribute.pc_relevant.none-task-blog-BlogCommendFromMachineLearnPai2-1.control&depth_1-utm_source=distribute.pc_relevant.none-task-blog- BlogCommendFromMachineLearnPai2-1.control
https://www.cnblogs.com/MWCloud/p/11237056.html