Copyright: Attribution, allow others to create paper-based, and must distribute paper (based on the original license agreement with the same license Creative Commons )
1, HDFS Ka构:
Name Node
Data Node
Secondary Name Node
1) The client reads the selected data. Name node and obtain the metadata.
2) Name node query metadata information. Metadata information on the two-dimensional databases such as mysql. Memory and disk were preserved copy.
3) returns the data to the Metadata Name node, and then returned to the client.
4) data proximity principle.
5) Name node level replication
2, the metadata store details of the Node the Metadata the Name
. 1) is stored in memory and disk each.
File name, file storage of a few, are stored in the storage position.
2) books similar to warehouse management.
NameNode (FileName, Replicas, Block-IDS, id2host)
/test/a.log,. 3, {blk_1, blk_2}, [{blk_1: [H0, h1 of, H2]}, {blk_2: [H0, h1 of, H2, h3]}]
3, the Name the Node
1) a major role.
Management stages of the entire file system. Maintenance of the entire file system file directory tree.
Listing each data block and the meta information file / directory corresponding. Accepting a user operation request.
2) document addresses stored in the linux file system.
fsimage metadata mirror file. Name node memory to store a certain period of metadata information.
edits: operation log file
fstime last checkpoint save time. ---- restore points similar systems.
-R & lt DFS -ls hdsf /
LS
CD tmp /
LS
CD DFS /
CD Current
. 3) the Name Node work characteristics:
memory storage metedat. For processing the read request.
When the write request over, name node editlog will first write to the disk, namely edits to write the log file, after a successful return,
will modify memory, returned to the client.
- V1.0 works and pseudo-distributed
Hadoop fsimage maintains a file, which is the metadata of the image file. But instead of saving the same.
Periodically update the contents of the file by merging edits. Secondary name node is used to merge
fsimage and edits the file to update Name Node of the metedata.
4, the Node Secondary the Name
HA
download fsimage and edits files, merge, push metedata Name Node's.
Workflow:
1) name node handover notification edits
2) and acquiring fsimage edits
. 3) will be loaded into memory and combined fsimage edits
. 4) to send the new name node fsiamge
. 5) replaces the old fsiamge new fsiamege
V1.0 name node and require separate deployment Secondary Name Node
1) client upload file, first transmission request information to the Node the Name
2) return the information to the Name Node Client
. 3) writes data to the Data Node.
4) Name Node requires edits operation log to record information success or failure.
If successful, edits +1
memory. 1 + The metaData
Fsiamge +1 No
5) time synchronization Fsianme:
first: checkpoint 3600 seconds
a second: maximum edits the file, once this value is exceeded, the checkpoint is forced to
5, data Node
provides real file data storage services.
1) HDFS Block default size is 128MB, 256MB file to Example
2) HDFs If a file is smaller than the size of a data block, the entire block of data does not take up storage space
3) Replication multiple copies. Three default. --- is a copy of the block