前言
- hadoop版本为2.9.2
- oracle java 1.8
- 三台centos7
- 官方文档
对于三台机器,编辑/etc/hosts,在最后加上下面几句话
10.61.8.247 master.wsxiot.cn
10.61.8.245 slave1.wsxiot.cn
10.61.8.246 slave2.wsxiot.cn
对于三台机器,编辑/etc/profile配置环境变量并重启
export JAVA_HOME=/root/platform/jdk1.8.0_201
export HADOOP_HOME=/root/platform/hadoop-2.9.2
export PATH=${JAVA_HOME}/bin:${HADOOP_HOME}/bin:${HADOOP_HOME}/sbin:$PATH
对于master,配置SSH免密登录
ssh-keygen -t rsa
ssh-copy-id master.wsxiot.cn
ssh-copy-id slave1.wsxiot.cn
ssh-copy-id slave2.wsxiot.cn
对于三台机器,${HADOOP_HOME}/etc/hadoop/hadoop-env.sh文件最后添加
export JAVA_HOME=/root/platform/jdk1.8.0_201
export HDFS_NAMENODE_USER=root
export HDFS_DATANODE_USER=root
export HDFS_SECONDARYNAMENODE_USER=root
对于三台机器,编辑${HADOOP_HOME}/etc/hadoop/core-site.xml
<configuration>
<property>
<name>hadoop.tmp.dir</name>
<value>/var/lib/hadoop</value>
</property>
<property>
<name>fs.defaultFS</name>
<value>hdfs://master.wsxiot.cn:9000</value>
</property>
</configuration>
对于三台机器,编辑${HADOOP_HOME}/etc/hadoop/hdfs-site.xml
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>
对于三台机器,编辑${HADOOP_HOME}/etc/hadoop/yarn-site.xml
<configuration>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.resourcemanager.hostname</name>
<value>master.wsxiot.cn</value>
</property>
</configuration>
对于三台机器,编辑${HADOOP_HOME}/etc/hadoop/mapred-site.xml
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>
对于三台机器,编辑${HADOOP_HOME}/etc/hadoop/slaves
slave1.wsxiot.cn
slave2.wsxiot.cn
启动HDFS,在master上,运行命令
hdfs namenode -format
start-dfs.sh
启动yarn,在master上,运行命令
start-yarn.sh
检查安装是否成功,每台机器使用jps命令,看下列节点是否启动起来
- mster: NameNode,Secondary NameNode,ResourceManager
- slave1: DataNode,NodeManager
- slave2: DataNode,NodeManager