centos--hadoop完全分布式

1,配置DNS
vi /etc/hosts
192.168.73.130 hadoop01
192.168.73.128 hadoop02
192.168.73.129 hadoop03
192.168.73.131 hadoop04

2.测试阶段关闭防火墙
#关闭服务
systemctl stop firewalld
#关闭开机自启动
systemctl disable firewalld

3.配置免密码登录

vi /etc/ssh/sshd_config

RSAAuthentication yes
           
PubkeyAuthentication yes
           
# 公钥文件路径          
           
AuthorizedKeysFile ~/.ssh/authorized_keys
如果是root用户 需要将root登录打开

ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
scp authorized_keys hadoop01:~/.ssh/
cat id_rsa.pub >> authorized_keys
复制到每台机器后追加 最后拷贝回四台机器

ssh 本身 如果有问题 拷贝到其他机器
ssh-add id_rsa
ssh-agent
eval $(ssh-agent)
ssh-add id_rsa
ssh-copy-id root@hadoop04

 

chmod 755 /root
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys
service sshd restart

4JAVA环境和Hadoop

cd /home
mv jdk-8u201-linux-x64/ java

tar -zxvf hadoop-3.2.0.tar.gz
mv hadoop-3.2.0 hadoop

vi /etc/profile
export JAVA_HOME=/home/java/jdk1.8.0_201
export JRE_HOME=${JAVA_HOME}/jre
export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib
export PATH=${JAVA_HOME}/bin:$PATH
export HADOOP_HOME=/home/hadoop
export PATH=$PATH:$HADOOP_HOME/bin
export PATH=$PATH:$HADOOP_HOME/sbin
export HADOOP_CONF_DIR=${HADOOP_HOME}/etc/hadoop

chmod 777 /home/java/jdk1.8.0_201/bin/java
source /etc/profile
验证java
java -version

cd /home/hadoop/etc/hadoop
vi hadoop-env.sh
vi mapred-env.sh
vi yarn-env.sh

export JAVA_HOME="/home/java/jdk1.8.0_201"

5. 修改Hadoop配置文件
按需配置Hadoop安装目录下的etc/hadoop目录中，需修改core-site.xml、hdfs-site.xml、mapred-site.xml、yarn-site.xml、workers文件，根据实际情况修改配置信息。

 

 附链接 https://www.cnblogs.com/shireenlee4testing/p/10472018.html