在ubuntu上搭建hadoop，记录一下
环境：ubuntu 16.04
hadoop-3.1.0
1.安装JDK
已经使用apt安装了jdk环境
whereis java
ls -l /usr/bin/java
ls -l /etc/alternatives/java
默认jdk路径：/usr/lib/jvm/java-8-openjdk-amd64
2.创建hadoop组、hadoop用户
sudo addgroup hadoop
sudo adduser --ingroup hadoop hadoop
给hadoop用户添加权限
sudo vi /etc/sudoers
添加如下：hadoop ALL=(ALL:ALL) ALL
3.配置SSH无密码登陆
切换用户
su hadoop
安装ssh，openssh-server
sudo apt install ssh
sudo apt install openssh-server
启动服务
sudo /etc/init.d/ssh start
查看服务是否启动
ps -e | grep ssh
进入宿主目录
cd /home/hadoop/
设置免密码登陆
ssh-keygen -t rsa -P ""
/home/hadoop/.ssh下生成两个文件： id_rsa （私钥） 、id_rsa.pub （公钥）
把公钥加到用于认证的公钥文件中
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
chmod 600 .ssh/authorized_keys
登入ssh，确认不需要密码
ssh localhost
登出
exit
4.下载解压hadoop
回到用户工作目录
cd ~
wget http://mirror.bit.edu.cn/apache/hadoop/common/hadoop-3.1.0/hadoop-3.1.0.tar.gz
tar -zxvf hadoop-3.1.0.tar.gz
移动到/usr/local/hadoop目录
sudo mv hadoop-3.1.0 /usr/local/hadoop
赋予用户对该文件夹的读写权限
sudo chmod 777 /usr/local/hadoop
5.配置
sudo vi ~/.bashrc
添加如下：
#HADOOP VARIABLES START
export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64/
export HADOOP_INSTALL=/usr/local/hadoop
export PATH=$PATH:$HADOOP_INSTALL/bin
export PATH=$PATH:$HADOOP_INSTALL/sbin
export HADOOP_MAPRED_HOME=$HADOOP_INSTALL
export HADOOP_COMMON_HOME=$HADOOP_INSTALL
export HADOOP_HDFS_HOME=$HADOOP_INSTALL
export YARN_HOME=$HADOOP_INSTALL
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_INSTALL/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_INSTALL/lib"
#HADOOP VARIABLES END
使配置生效
source ~/.bashrc
sudo vi /usr/local/hadoop/etc/hadoop/hadoop-env.sh
添加如下：
export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
6.WordCount测试
执行hadoop自带实例WordCount，验证是否安装成功
cd /usr/local/hadoop
创建输入文件
mkdir input
cp README.txt input
执行
bin/hadoop jar share/hadoop/mapreduce/sources/hadoop-mapreduce-examples-3.1.0-sources.jar org.apache.hadoop.examples.WordCount input output
查看字符统计结果
cat output/*
参考：
https://blog.csdn.net/wenyun_kang/article/details/77413714