Detailed steps for pseudo-distributed installation of Hadoop

1. Install JDK
2. Turn off the firewall
   service iptables stop
   chkconfig iptables off
3. Turn off the boot initialization of the cloud host
   service cloud-init stop
   chkconfig cloud-init off
   service cloud-init-local stop
   chkconfig cloud-init-local off
4. Modify the host name
   vim /etc/sysconfig/network to
   modify the value of HOSTNAME, for example, modify HOSTNAME to hadoop01. After the
   change, save and exit, and re-validate
   source /etc/sysconfig/network
5. Map the host name and IP
   vim /etc/hosts
   add IP host name
   save and exit
6. Reboot
   reboot
7. Password
   - free login Generate key: ssh-keygen — press Enter all the way, no need to enter any data
   Copy public key: ssh-copy-id root@hadoop01
   Test: ssh hadoop01 If you don’t need any password after press enter, it means the secret-free is successful Then enter logout
8. Download the Hadoop installation package
   cd /home/software/
   wget http://bj-yzjd.ufile.cn-north-02.ucloud.cn/hadoop-2.7.5.tar.gz
9. Unzip
   tar -xvf hadoop-2.7.5.tar.gz
10. Enter the subdirectory
    cd hadoop-2.7.5/etc/hadoop/
11. Edit
    vim hadoop-env.sh
    modify the properties
    export JAVA_HOME=/home/presoftware/jdk1.8
    export HADOOP_CONF_DIR=/home/software/hadoop-2.7.5/etc/hadoop
    save and exit, re-validate
    source hadoop-env.sh
12. Edit
    vim core-site.xml to
    add
    
    fs.defaultFS
    hdfs://hadoop01:9000
    
    
    hadoop.tmp.dir
    /home/software/hadoop-2.7.5/tmp
    
    save and exit
13. Edit
    vim hdfs-site.xml to
    add
    
    dfs.replication
    1
    
    save and exit
14. Edit
    cp mapred-site.xml.template mapred-site.xml
    vim mapred-site.xml to
    add
    
    mapreduce.framework.name
    yarn to
    
    save and exit
15. Edit
    vim yarn-site.xml to
    add
    
    yarn.resourcemanager.hostname
    hadoop01
    
    
    yarn.nodemanager.aux-services
    mapreduce_shuffle
    
    save and exit
16. Edit
    vim slaves to
    delete localhost, add the current host name,
    save and exit
17. Modify the environment variable
    vim /etc/profile
    and append
    export HADOOP_HOME=/home/software/hadoop-2.7.5
    export PATH= PATH: PATH: at the end of the file$PATH:$ HADOOP_HOME/bin:$HADOOP_HOME/sbin
    save and exit, re-validate
    source /etc/profile
18. Format Hadoop
    hadoop namenode -format,
    if Storage directory /home/software/hadoop-2.7.5/tmp/dfs/name has been successfully formatted, it means that the formatting is successful
19. Start Hadoop
    start-all.sh
20. View the process through jps
    Jps
    NameNode port: 50070
    DataNode port: 50075
    SecondaryNameNode port: 50090
    ResourceManager port: 8088
    NodeManager

Common errors:
1. If Name or Service not known or Unknown Host appears, then the host name is configured incorrectly, check the /etc/hosts file
2. If Command not found appears, then the environment variable is configured incorrectly, or there is no source after the configuration is complete
3. If NameNode/DataNode/SecondaryNameNode is missing, then check core-site.xml/hdfs-site.xml is configured correctly
4. If ResourceManager/NodeManager is missing, then check mapred-site.xml/yarn-site.xml Configured correctly