Foreword
1. Installation Configuration JDK
1. Download JDK
Click Download to download
Click Accept License Agreement before downloading, and then download jdk-12_linux-64_bin.tar.gz
2. Extract JDK
Download directory, open terminal, enter
tar zxvf jdk-12_linux-64_bin.tar.gz
After extracting the file folder to / usr / local folder, enter the following at the command line
sudo mv jdk-12_linux-64_bin / usr / local
jdk-12_linux-64_bin file after you unpack the folder, and if you are not the same, clear modified according to the actual situation.
3. Configure Java environment
Enter the following command terminal
sudo gedit ~ / .bashrc
End of the file is written in the following
Export the JAVA_HOME = / usr / local / JDK-12_linux-64_bin
Export the JRE_HOME the JAVA_HOME = $ {} / JRE
Export the CLASSPATH =:. $ {The JAVA_HOME} / lib: $ {} the JRE_HOME / lib
Export the PATH = $ {JAVA_HOME} / bin: $ PATH
Then enter the following command terminal of
source ~ / .bashrc
4. Test Java installation was successful
Enter the following command terminal
java -version
If the configuration is successful, it will show the version of java, and then continue to enter the following command
javac
Configuration successful will show commands you can use
2. Download hadoop
Click Download to download
Download hadoop-2.7.6.tar.gz this version, there is a need to download other versions
3. Extract the / opt directory (if there is a need to switch to another directory, the latter operation should continue to modify)
Open the terminal to the download directory, Run
tar -zxvf hadoop-2.7.6.tar.gz -C / opt /
4. Configuration Environment Variables hadoop
Open a command line, enter the following command
sudo gedit / etc / profile
Add the following code in the file
Export HADOOP_HOME is = / opt / Hadoop-2.7.6
Export the PATH the PATH = $: $ HADOOP_HOME is / bin
Run the following command line in the
source / etc / profile
5. Configuration hadoop
5.1 Configuration hadoop-env.sh
Run the following commands in the command line
sudo gedit /opt/hadoop-2.7.6/etc/hadoop/hadoop-env.sh
Find # The java implementation to use it the following line to read:.
Export JAVA_HOME = / usr / local / the JDK-12_linux-64_bin
If you do not follow my steps above to install java, java clean fill out your own path
5.2 Configuring core-site.xml (5.2 and 5.3 configuration file path and file the port with their own habits configuration)
Where IP: 192.168.44.128 virtual machine ip, can not be set to localhost, if localhost, connected hive will be reported on the server (virtual machine) anomalies on the back windows with saprk
ifconfig see your ip address at the command line, in the following code 192.168.44.128 change your ip on it
Enter the following at the command line
sudo gedit /opt/hadoop-2.7.6/etc/hadoop/core-site.xml
在打开的文件中添加如下内容
<configuration>
<property>
<name>hadoop.tmp.dir</name>
<value>file:///opt/hadoop-2.7.6</value>
<description>Abase for other temporary directories.</description>
</property>
<property>
<name>fs.defaultFS</name>
<value>hdfs://192.168.44.128:8888</value>
</property>
</configuration>
Save and close the file, enter the following command at the command line
sudo gedit /opt/hadoop-2.7.6/etc/hadoop/hdfs-site.xml
在打开的文件中修改如下内容
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>file:///opt/hadoop-2.7.6/tmp/dfs/name</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>file:///opt/hadoop-2.7.6/tmp/dfs/data</value>
</property>
</configuration>
6.SSH free secret landing
Enter the following command line
sudo APT-GET-install OpenSSH Server
cd ~ / .ssh /
SSH SSH-keygen -t rsa localhost
/ * this process continued to press Enter on it * /
CAT id_rsa.pub >> authorized_keys
7. Start and stop
Hdfs first start needs to be formatted, enter the following at the command line (that asks input Y or N, Y can lose all)
CD /opt/hadoop-2.7.6
./bin/hdfs NameNode -format
Start
./sbin/start-dfs.sh
Stop
./sbin/stop-dfs.sh
Verification, browser and enter: http: //192.168.44.128: 50070
Simple validation hadoop command:
hadoop FS -mkdir / the Test
In the browser view, as shown in the figure below, that is success
You can also enter the following command to upload the file to the hadoop
hadoop FS -put / the Test 1.txt / the Test
8. Configuration yarn
8.1 Configuration mapred-site.xml
Command line, enter the following:
CD /opt/hadoop-2.7.6/etc/hadoop/
CP-site.xml.template mapred-mapred the site.xml
the sudo the gedit-mapred the site.xml
Adding content in the file
<the Configuration>
<-! Notification framework MR use YARN ->
<Property>
<name> mapreduce.framework.name </ name>
<value> the Yarn </ value>
</ Property>
</ the Configuration >
8.2 Configuration yarn-site.xml
Enter the command line command as follows:
the sudo the gedit Yarn-the site.xml
Add the contents in the file
<Configuration>
<-! Mode of access to data reducer is mapreduce_shuffle ->
<Property>
<name> yarn.nodemanager.aux-Services </ name>
<value> mapreduce_shuffle </ value>
</ Property >
</ Configuration>
8.3 yarn start and stop
Start
cd /opt/hadoop-2.7.5
./sbin/start-yarn.sh
Stop
./sbin/stop-yarn.sh
Browser View: http://192.168.44.128:8088
jps view the process
Output as shown, it indicates the success of the configuration standalone mode hadoop