1 HDFS write data flow
1 .1 analysis of documents written
HDFS write data flow as shown in FIG.
1 ) the client through the Distributed FileSystem module to request NameNode upload files, whether NameNode check the target file already exists, the parent directory exists.
2 ) the NameNode return if it can be uploaded.
3 ) client requests the first Block to upload to several DataNode on the server.
. 4 ) the NameNode returns . 3 th DataNode nodes, respectively, dn1, dn2, dn3.
5 ) the client by requesting dn1 upload data FSDataOutputStream module, dn1 receipt of the request will continue to call dn2, then dn2 call dn3, this communication channels established.
6 ) DN1, DN2, DN3 step by step answer clients.
7) The client first began to upload dn1 Block (starting with disk reads data into a local memory cache) to Packet units, dn1 receive a Packet will pass dn2, dn2 passed DN3; dn1 each pass a packet places a reply queue waiting for a reply .
8) When a Block after the transfer is complete, the client requests again NameNode Block second upload server. (Repeat perform steps 3-7).
. 1 .2 network topology - node distance calculation
HDFS in the process of writing data, the NameNode can choose from the most recent data to be uploaded from the DataNode receive data. Well, this recently from how to calculate it ?
Node distance: two nodes to the nearest common ancestor distance sum.
例如，假设有数据中心d1机架r1中的节点n1。该节点可以表示为/d1/r1/n1。利用这种标记，这里给出四种距离描述，如上图所示。
大家算一算每两个节点之间的距离，如下图所示。
1.3 机架感知（副本存储节点选择）
1. 官方ip地址
For the common case, when the replication factor is three, HDFS’s placement policy is to put one replica on one node in the local rack, another on a different node in the local rack, and the last on a different node in a different rack.
2. Hadoop2.7.2副本节点选择
4.2 HDFS读数据流程
HDFS的读数据流程，如图所示。
1）客户端通过Distributed FileSystem向NameNode请求下载文件，NameNode通过查询元数据，找到文件块所在的DataNode地址。
2）挑选一台DataNode（就近原则，然后随机）服务器，请求读取数据。
3）DataNode开始传输数据给客户端（从磁盘里面读取数据输入流，以Packet为单位来做校验）。
4）客户端以Packet为单位接收，先在本地缓存，然后写入目标文件。