The series Overview: Hadoop3.1.1 system architecture - design principles set forth Detailed graphics and Client Source: Overview
About RPC (Remote Procedure Call), if there is no concept, you can refer to RMI (Remote Method Invocation).
Briefly, a network transmission package details, and the user an idea of the proxy object is exposed.
We know NameNode and clients are generally not on the same machine, the client (Client) calls NameNode way through RPC, get on NameNode
File directory structure, block mapping information, file permissions, file location (Locations, that is, the information DataNode) etc. information resources.
Below will explain when we call FileSystem.create, FileSystem.append, FileSystem.open and other methods, to prepare for data transmission to do.
These methods are the most important methods, other methods of preparation phase and their preparation phase almost.
1 .FileSystem.create ready built environment:
The figure is the method we call FileSystem.create when namenode place on the final call to.
FIG namenode seen here is the Proxy a (proxy), the real object is actually encapsulated agent.
According to JDK dynamic proxy commonly used in real object -> Handler-> Proxy $
Package relationships, we can see that the agent system, the agent being a first layer (red box) is an object ClientNamenodeProtocolTranslatorPB
ClientNamenodeProtocolTranslatorPB class may be an operation to NameNode create, append, delete, etc., but in fact he is only one shell
The actual remote call or Invoker (blue box) ProtobufRpcEngine second layer.
Let us look after the call Filesystem create, append, open and other methods, Client side will get what, and what to do.
When we call Filesystem.create, the call is namenode create method.
If NameNode create a file in the directory structure of the metadata is successful, will create the file information is returned to the client
We can see all kinds of information files created from figures such as
File permissions: rw-r - r-- (owner files for read and write for the owner where the group members and other human-readable)
Backup: 3
The first data block information file ......
Noteworthy is the file block (Block) size blocksize = 134217728 bytes
We show that the size of a block is 128MB. 128MB is the default for most large file transfers scene.
According file metadata information stat, we created a DFSOutputStream (simplicity will DFSStripedOutputStream with DFSOutStream as a discussion, in fact, the former inherited the latter)
DFSOutputStream very important, FSDataOutputStream object of our operations in the packaging of his objects.
That is our function call is actually DFSOutputStream responsible for implementation.
And he is responsible for opening and maintaining the thread guard DataStreamer manage our data transmission
(After the code is the process illustrated in FIG.)
So far, the preparatory stage of completion of the create method. DataStreamer is turned on, waiting for data to be transmitted.
2 .FileSystem.append ready built environment:
Call FileSystem.append, eventually calling on namenode.append, that is, RPC calls, and similar to the above principle.
We note here wrote while (true) death cycle, mainly if NameNode busy, no time to deal with our request, it will throw an exception
Endless loop allows the program to generate an exception when will sleep after a period of time to try again, rather than exit.
如果请求超过了一定时间（5秒），那么将会抛出异常，停止重试。
较为重要的是函数的返回值
我们发现我们从namenode.append那里拿到的东西，有二
一是我们请求append的文件的最后一个数据块的信息
二是我们请求append的文件他本身的信息
比如我们.append("abc/abc");
那么lastBlock返回的是组成abc/abc这个文件的所有数据块中，最后那个数据块的信息。
fileStatus是abc/abc这个文件他本身的信息
我们看看这些信息包括了什么。
最后一个数据块的信息中，比较重要的是存储这个数据块的位置，位置信息中包含DataNode的IP和端口，根据位置信息，我们可以建立起对块进行读写的流水 线（pipeLine）
fileStatus包含文件的权限，所有者等等关于文件的信息。
和.create方法一样，我们在获取文件信息后也利用文件信息创建了一个DFSOuputStream，并且在DFSOutputStream构造完后也开启了DataStreamer(DataStreamer的创建和启动不在下图中)
至此，.append的准备阶段完成。
3.open函数做的准备;
当我们调用FileSystem.open , 最终会调用到DFSClient的callGetBlockLocations函数，这个函数通过调用namenode的getBlockLocations获得
数据块的存储位置，我们看看他返回了什么。
一.我们打开只有一个Block的文件
1.我们注意到，blocks（组成文件的数据块），确实只有一个
2.并且lastLocatedBlock(组成文件的最后一个数据块)，offset是0，因为只有一个数据块，所以offset是0，等会我们打开有两个数据块的文件，这个值不是0.
3.每个block都有其所在DataNode的位置信息，比较重要的是IP和端口，可以通过他们建立文件读写的流水线
二，我们打开有两个数据块的文件
值得注意的是，确实有两个数据块，并且lastLocatedBlock就是第二个数据块（因为只有两个数据块，最后一个数据块当然是第二个数据块）
并且第二个数据块的offerset是134217728（字节），换算后就是128MB, 是较新Hadoop默认的一个Block大小，
我们用文件大小fileLength减去一个数据块的大小 185516505 - 134217728 = 51298777 ,正好是第二个数据块的大小。
获取到关于文件数据块的信息后，调用DFSClient.openInternal
openInternal会创建DFSInputStream
不像DFSOutputStream , DFSInputStream的创建并不会创建DataStreamer守护线程并启动，因为不需要。
至此，三大开启准备阶段的方法的讲解完毕。