Because of the needs of the company's project, we started to build a Hadoop cluster environment. In this process, the main reference is http://blog.csdn.net/pucao_cug/article/details/71698903. Many thanks to the author of this article.
Basically, there is no problem going down this way. Just to add some caveats here.
platform:
JDK: 1.8.60
Hadoop: 2.8.3.
Centos 7 64-bit: 3.
Points to pay attention to.
1. The hosts in Centos7 should be configured well, otherwise, the datanode will not be able to connect to the namenode, and the cluster will not be formed.
2. During the process of building a hadoop cluster, the same configuration was used when configuring the xml files of the three nodes. Some people asked if it was the same. In the three nodes, the resource manager and the namenode use the name of the master. The slave file is written with the host name of the slave.
When it is found that the datanode cannot connect to the namenode, some commands can be used to help find the problem. I use the following command to see if the port is visible.
At the same time, I use the TCPing tool to check whether a certain port of the namenode can be connected from the outside.