1.jdk安装
参考:https://blog.csdn.net/qq_37497275/article/details/81050968
2.配置SSH免密登陆
(1) 安装ssh sudo apt-get install ssh
(2) 安装rshync sudo apt-get install rshync
(3) 安装服务端 sudo apt-get install openssh-server
(4) 用命令登入ssh localhost
Welcome to Ubuntu 18.04 LTS (GNU/Linux 4.15.0-29-generic x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
System information as of Sat Jul 28 04:53:46 UTC 2018
System load: 0.01 Processes: 162
Usage of /: 34.0% of 19.56GB Users logged in: 1
Memory usage: 26% IP address for ens33: 192.168.186.129
Swap usage: 0%
* Introducing Minimal Ubuntu for docker and clouds. 30 MB base image and
optimised kernels on public clouds. Made for machines and containers.
- https://bit.ly/minimal-ubuntu
* Canonical Livepatch is available for installation.
- Reduce system reboots and improve kernel security. Activate at:
https://ubuntu.com/livepatch
75 packages can be updated.
3 updates are security updates.
Last login: Sat Jul 28 04:33:49 2018 from 192.168.186.1
(5)设置免密登陆
退出ssh : exit
1.生成公钥私钥 ssh -keygen -t rsa,将在~/.ssh文件夹下生成文件id_rsa:私钥,id_rsa.pub:公钥
2.导入公钥到认证文件,更改权限:
1)导入本机:cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
2)导入服务器:
首先将公钥复制到服务器:
scp ~/.ssh/id_rsa.pub xxx@host:/home/xxx/id_rsa.pub
然后,将公钥导入到认证文件,这一步的操作在服务器上进行:
cat ~/id_rsa.pub >> ~/.ssh/authorized_keys
最后在服务器上更改权限:
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys
3)测试:ssh localhost 第一次需要输入yes,之后就不需要了。
3 Hadoop安装
在官网上下载 安装包hadoop-3.1.0.tar.gz
在 /usr/local/hadoop下面解压
进入解压后的文件hadoop-3.1.0
创建input : sudo mkdir input
执行:cp cp README.txt input
执行:bin/hadoop jar share/hadoop/mapreduce/sources/hadoop-mapreduce-examples-2.7.3-sources.jar org.apache.hadoop.examples.WordCount input output
hadoop单机配置(非分布式模式)
hadoop默认是非分布式模式,不需要进行其它配置。可以测试demo来观察是否配置正确。
Hadoop默认不会覆盖结果文件,因此再次运行上面实例会提示出错,需要先将output删除。
如果加入配置文件中
4.修改配置文件,和JDK的安装一样,可以选择修改哪个文件。这里修改/etc/profile
export HADOOP_HOME=/usr/local/hadoop/hadoop-3.1.0
export PATH=.:${JAVA_HOME}/bin:${HADOOP_HOME}/bin:$PATH
source /etc/hadoop