hadoop learning:
1: Prepare the Linux environment
1" Build a virtual machine under win10
2" Rent a cloud host, such as Alibaba Cloud/Tencent Cloud, etc.
2: Install JDK and set environment variables
3: Install Hadoop Configure Hadoop
Prepare the linux environment
Tencent Cloud:
vim is a powerful text editor under ubuntu:
Enter vim on the command line to prompt whether to install,
If it is not installed, you need to enter sudo apt install vim-gtk
Install JDK:
First determine whether it has been installed, enter javac on the command line, there will be a prompt;
If not installed, then: suto apt install openjdk-8-headless
See where it is installed: dpkg -L openjdk-8-headless
Then configure the environment variables:
vim /etc/profile
export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 //After the equal sign is the jdk installation path
export JRE_HOME=$JAVA_HOME/jre
export CLASSPATH=$JAVA_HOME/lib:$JRE_HOME/lib:$CLASSPATH
export PATH=$JAVA_HOME/bin:$JRE_HOME/bin:$PATH
Note: Press the insert key to start editing, and press the esc key to exit editing;
Then you can exit the text editor,
Exit without saving :q!
save and exit :wq
save without exit :w
If prompted to read only, do not allow editing: sudo vim /etc/profile
ok!
Let the jdk environment variables take effect:
source /etc/profile
Configure hadoop:
wgit : Download files using wget
Download hadoop:
wget http://mirrors.shu.edu.cn/apache/hadoop/common/hadoop-2.6.5/hadoop-2.6.5.tar.gz
Unzip:
tar -xvzf hadoop-2.6.5.tar.gz
Configure environment variables:
sudo vim /etc/profile
export HADOOP_HOME=usr/soft/hadoop-2.6.5
export PATH=$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$PATH
verify:
hadoop version