hadoop cluster windows linux installation and deployment

1. JDK1.8 installation (no explanation) 

2. Download hadoop2.7.5.tar.gz , or have to go to Baidu download. 

3. Download hadooponwindows-master.zip (useful for later)

Install Hadoop:
After downloading the archive of good Hadoop2.7.5, extract to a directory, I put D: \ hadoop-2.7.5

4 hadoop modify configuration files

• 1. Edit "D: \ etc hadoop-2.7.5 \ \ hadoop" core-site.xml file, paste the following text into it, and save it;

<configuration>
    <property>
        <name>hadoop.tmp.dir</name>
        <value>/D:/hadoop-2.7.5/workplace/tmp</value>
    </property>
    <property>
        <name>dfs.name.dir</name>
        <value>/D:/hadoop-2.7.5/workplace/name</value>
    </property>
    <property>
        <name>fs.default.name</name>
        <value>hdfs://localhost:9000</value>
    </property>
</configuration>

• 2. Edit "D: \ hadoop-2.7.5 \ etc \ hadoop" mapred-site.xml directory (mapred-site.xml.template will not rename mapred-site.xml) file, paste the contents of what and save:

<configuration>
    <property>
       <name>mapreduce.framework.name</name>
       <value>yarn</value>
    </property>
    <property>
       <name>mapred.job.tracker</name>
       <value>hdfs://localhost:9001</value>
    </property>
</configuration>

• 3. Edit "D: \ hadoop-2.7.5 \ etc \ hadoop" hdfs-site.xml file in the directory, paste the following content and save it. Please create your own data directory, where I create a workplace / data directory under HADOOP_HOME catalog:

<the Configuration>
    <-! this parameter is set to 1, because it is a stand-alone hadoop ->
   

 <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>
    <property>
        <name>dfs.data.dir</name>
        <value>/D:/hadoop-2.7.5/workplace/data</value>
    </property>
</configuration>

• 4. Edit "D: \ hadoop-2.7.5 \ etc \ hadoop" yarn-site.xml file in the directory, paste the following content and save it;

<configuration>
    <property>
       <name>yarn.nodemanager.aux-services</name>
       <value>mapreduce_shuffle</value>
    </property>
    <property>
       <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
       <value>org.apache.hadoop.mapred.ShuffleHandler</value>
    </property>
</configuration>

• 5. Edit "D: \ hadoop-2.7.5 \ etc \ hadoop" hadoop-env.cmd files in the directory, with the JAVA_HOME commented @rem, edit JAVA_HOME path, and save:

@rem set JAVA_HOME=%JAVA_HOME%

set JAVA_HOME=D:\java\jdk

• Replace file

The downloaded hadooponwindows-master.zip (note the first step in the download address, do not know can go to the beginning of demand notes column view) unpack all of the files in the bin directory of the unzipped directly covered bin directory of Hadoop.

5 Operating Environment

• 1. Run cmd window, perform " HDFS NameNode -format continue the entry 'y' rear", as shown:

• 2. Run the cmd window, switch to the hadoop sbin directory , the implementation of " Start-all.cmd ", it will start the following process.

After successful, as shown: 

 

 

Currently Hadoop is now fully set.

 

Test 6:

Depending on your core-site.xml configuration, then you can: hdfs: // localhost: 9000 to operate on a hdfs

 

hadoop own web console GUI

• 1. Explorer the GUI: HTTP: // localhost: 8088 / ; (default port 8088, the port that they can change at will)

• 2. Node Manager the GUI: HTTP: // localhost: 50070 / ;