Due to learning needs, if all Hadoop-related software is installed on the virtual machine, it will be very stuck due to the low configuration of the computer, so
connect the Hadoop installed on the virtual machine CentOS7 through the eclipse-hadoop plug-in on windows
Java version on the virtual machine: jdk-7u75-linux-x64
Hadoop version on the virtual machine: hadoop-2.6.0
1. Hadoop installation on CentOS
The operating system of the reference document is Ubuntu, but CentOS is basically the same,
you can directly use your own root user, no need to create a user
2. Win10eclipse install Hadoop plug-in
The reference document is installed on Linux, and it is similar to Windows. Just pay attention to the plug-in version corresponding to the Hadoop version. An
error may appear when installing the plug-in:
Could not create the view: org.apache.hadoop.eclipse.view.servers
My solution is Directly delete the .metadata file in the workspace, refer to
3.eclipse connects to hadoop through hadoop plug-in
3.1 Connect directly through the virtual machine address
Location name: The connection name is optional.
Host: Fill in the IP address of my virtual machine ( 这需要在Hadoop的配置中将localhost改为虚拟机的IP地址，代码中的路径也需要用IP地址表示，用localhost会报错)
Port under Map/Reduce (V2) Master: MapReduce port number defaults to 50020
Port under DFS Master: HDFS port number defaults to 9000
User name: User name connected to CentOS
This parameter refers to the source of the picture
Other references
3.2 Forward port connection through VMware virtual network editor
Inspired by this when searching for bugs , use this method. But it feels like it doesn't make much sense.
VMware port forwarding setting
Host: 127.0.0.1 is the local host
Host: fill in the IP address of my virtual machine (this needs to be changed from localhost to the virtual machine’s IP address in the Hadoop configuration)
under Map/Reduce(V2) Master Port: This port is relatively random, as long as it is not occupied.
Port under DFS Master: It is also fine if it is not occupied. If the port is occupied, an error will be reported when running
(check the PID corresponding to the occupied port 49157, and enter the command: netstat -aon|findstr "49157")
User name: current windows user name
The parameter settings are correct, and after confirming that Hadoop on CentOS is started, connect to
the error:
After inquiry, some say that this is because the previous configuration is incorrect, and some say that this is because there are no files in HDFS, but I am not for these reasons. Network problem The connection problem is solved, and the connection can be normal. After checking, if there is no problem with the relevant configuration and the operation is normal, the error can be ignored.
There is a problem:
After querying, it is found that the problem is caused by the firewall of CentOS7 not being closed, and the configuration of hdfs may be changed at the same time