A master-slave Hadoop deployment (2)

Deploying a Hadoop from a master (2)

---

1. Turn off the firewall and Linux daemon,
execute the command:

iptables -F
setenforce 0

2. Hadoop cluster is initialized, execute commands on namenode (host)
hdfs namenode -format

3. Start Hadoop, Run the following commands on NameNode (host)
start-all.sh

4. The master and slave execution command jps, check whether the normal start cluster, the results shown
Host:

Slave:

5. Hadoop some simple operation process:
create a directory on bigdata hdfs, and upload a file directory wordcount_test

hadoop fs -mkdir /bigdata
hadoop fs -put /root/wordcount_test /bigdata

6. Use of the shell interface to perform Wordcount Hadoop

hadoop jar /root/hadoop-2.8.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.8.0.jar wordcount /bigdata/wordcount_test /output

Notes : 1 / output this file output directory must not be created in advance, otherwise it will error
2 Do wordcount to allocate enough memory, or you'll get stuck (I allocated 4G), this yarn-site.xml configuration in this file, otherwise it will stuck, as follows:

      <property>
        <name>yarn.nodemanager.resource.memory-mb</name>
        <value>4096</value>
  </property>

7. Check whether the successful
execution commandhadoop fs -ls /output

8. Review the results of
the implementation of the commandhadoop fs -cat /output/part-r-00000

So far, Hadoop set up has been completed, and the use of Hadoop to complete a simple Wordcount applet.

to sum up:

Be sure to turn off the firewall in advance, or to upload files hdfs error.
Be sure to allocate enough memory, otherwise MapReduce will get stuck.