Linux cluster installation and set up Hadoop3.1.2 (4) - Installation JDK1.8 and hadoop3.1.2

Chapter 5: Install JDK1.8 and hadoop3.1.2

5.1 create a folder in the / opt directory

(1) Create a module in the / opt directory, software folder

sudo mkdir module
sudo mkdir software

(2) modify the module, the owner of the cd software folder

sudo chown zpark:zpark module/ software/

Install JDK 5.2

1. Uninstall the existing JDK
2. (1) check whether the installed Java software:

 [zpark@hadoop104 opt]$ rpm -qa | grep java 

(2) if the installed version is lower than 1.7, uninstall the JDK:

 [zpark@hadoop104 opt]$ sudo rpm -e 软件包

(3) View JDK installation path:

  [zpark@hadoop104 ~]$ which java

3. Installation mobaxterm, with mobaxterm tool JDK directory opt into the software folder, you can install all Next

https://.mobatek.net/download.html

4. Select jdk1.8 and hadoop3.1.2 drag:
5. Opt directory in the Linux system to see whether the import was successful package

[zpark@hadoop104 opt]$ cd software/
[zpark@hadoop104 software]$ ls 

`
6. JDK to the decompression / opt / module directory

[zpark@hadoop104 software]$ tar -zxvf jdk-8u181-linux-x64.tar.gz -C /opt/module

8. Configuring JDK environment variables
   (1) to obtain the JDK path

 [zpark@hadoop104 jdk1.8.0_181]$ pwd /opt/module/jdk1.8.0_181 	

(2) Open the / etc / profile file

[zpark@hadoop104 software]$ sudo vi /etc/profile

Add JDK path in the profile end of the file

#JAVA_HOME 
export JAVA_HOME=/opt/module/jdk1.8.0_181 
export PATH=$PATH:$JAVA_HOME/bin   	

(3) save and exit: WQ
(4) file after the changes to take effect

[zpark@hadoop104 jdk1.8.0_181]$ source /etc/profile

9. JDK is installed successfully test

[zpark@hadoop104 jdk1.8.0_181]# java -version 
java version "1.8.0_181"

Note: Restart (if you can not java -version restart)

[zpark@hadoop104 jdk1.8.0_181]$ sync 
[zpark@hadoop104 jdk1.8.0_181]$ sudo reboot

5.3 install Hadoop

Hadoop Download:

https://archive.apache.org/dist/hadoop/common/hadoop-3.1.2/

With mobaxterm tool hadoop-3.1.2.tar.gz opt into the directory software folder, the installation package into the path Hadoop

 [zpark@hadoop104 ~]$ cd /opt/software/

File to extract the installation / opt / module below

[zpark@hadoop104 software]$ tar -zxvf hadoop-3.1.2.tar.gz -C /opt/module/

Check whether the decompression success

 [zpark@hadoop104 software]$ ls /opt/module/ hadoop-3.1.2

Hadoop environment variable added to
(1) obtain the installation path Hadoop

 [zpark@hadoop104 hadoop-3.1.2]$ pwd /opt/module/hadoop-3.1.2 	

(2) Open the / etc / profile file

 [zpark@hadoop104 hadoop-3.1.2]$ sudo vi /etc/profile 

Add JDK end of the path in the profile file:

##HADOOP_HOME 
export HADOOP_HOME=/opt/module/hadoop-3.1.2 
export PATH=$PATH:$HADOOP_HOME/bin 
export PATH=$PATH:$HADOOP_HOME/sbin 

(3) save and exit: WQ
(4) file after the changes to take effect

 [zpark@hadoop104 hadoop-3.1.2]$ source /etc/profile

Test whether the installation is successful

[zpark@hadoop104 hadoop-3.1.2]$ hadoop version 
Hadoop 3.1.2

Restart (if Hadoop command can not restart)

[zpark@hadoop104 hadoop-3.1.2]$ sync
[zpark@hadoop104 hadoop-3.1.2]$ sudo reboot

Hadoop directory structure
1, see the directory structure Hadoop

[zpark@hadoop104 hadoop-3.1.2]$ ll
总用量 52
drwxr-xr-x. 2 zhangyong zhangyong  4096 5月  22 2017 bin
drwxr-xr-x. 3 zhangyong zhangyong  4096 5月  22 2017 etc
drwxr-xr-x. 2 zhangyong zhangyong  4096 5月  22 2017 include
drwxr-xr-x. 3 zhangyong zhangyong  4096 5月  22 2017 lib
drwxr-xr-x. 2 zhangyong zhangyong  4096 5月  22 2017 libexec
-rw-r--r--. 1 zhangyong zhangyong 15429 5月  22 2017 LICENSE.txt
-rw-r--r--. 1 zhangyong zhangyong   101 5月  22 2017 NOTICE.txt
-rw-r--r--. 1 zhangyong zhangyong  1366 5月  22 2017 README.txt
drwxr-xr-x. 2 zhangyong zhangyong  4096 5月  22 2017 sbin
drwxr-xr-x. 4 zhangyong zhangyong  4096 5月  22 2017 share

2, important directories
(1) bin directory: store the script for Hadoop-related services (HDFS, YARN) operate
(2) etc directory: Hadoop configuration file directory, stored Hadoop configuration file
(3) lib directory: store for Hadoop local library (data compression and decompression functions)
(4) sbin directory: start or stop the script storage Hadoop related services
(5) share directory: storage dependent jar package, documents, and official cases of Hadoop