hadoop2.7.5分布式集群安装(Hdoop2.7.5)

三台虚拟机

192.168.79.132 master
192.168.79.133 slave1

192.168.79.134 slave2

1.修改主机名

hostnamectl set-hostname (name)

2.配置主机的映射(每台都要设置)

vi /etc/hosts

3.设置免密码登陆

ssh-keygen

输入上面的命令后，按三次回车即可

将密钥复制到其他机器（包括本机）

ssh-copy-id master

ssh-copy-id slave1

ssh-copy-id slave2

在复制密钥时会提示是否复制输入 yes 并且要输入密码

每台机器都要复制

4.解压java和hadoop

tar -zxvf jdk-8u77-linux-x64.tar.gz -C /opt

tar -zxvf hadoop-2.7.5.tar.gz -C /opt

-C 指定解压到的目录

5.修改文件夹的名称(方便管理)

mv /opt/hadoop-2.7.5 /opt/hadoop

mv /opt/jdk1.8.0_77 /opt/jdk

6.配置环境变量

vi /etc/profile

在文件末尾加入

export JAVA_HOME=/opt/jdk

export HADOOP_HOME=/opt/hadoop

export PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin

source /etc/profile 使环境变量生效

验证

java -version

hadoop version

7.修改hadoop的配置文件

切换到hadoop的配置文件目录

cd /opt/hadoop/etc/hadoop/

vi hadoop-env.sh

export JAVA_HOME=/opt/jdk

vi slaves

slave1

slave2

vi core-site.xml

<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://master:9000</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/opt/hadoop/tmp</value>
</property>
</configuration>

vi hdfs-site.xml

<configuration>

<property>
<name>dfs.namenode.name.dir</name>
<value>/opt/hadoop/name</value>
</property>


<property>
<name>dfs.replication</name>
<value>2</value>
</property>

<property>
<name>dfs.datanode.data.dir</name>
<value>/opt/hadoop/data</value>
</property>
</configuration>

vi mapred-site.xml

<configuration>

<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>`
</configuration>

vi yarn-site.xml

<configuration>



<property>
<name>yarn.resourcemanager.hostname</name>
<value>master</value>
</property>


<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>

</property>

</configuration>

8.将hadoop和java和/etc/profile复制到另外两台机器

scp -r /opt/jdk /opt/hadoop @slave1:/opt/

scp -r /opt/jdk /opt/hadoop @slave2:/opt/

scp -r /etc/profile @slave1:/etc/profile

scp -r /etc/profile @slave2:/etc/profile

PS：在slave1和slave2上要使环境变量生效

source /etc/profile

9.启动hadoop

在启动之前要格式化hadoop

hdfs namenode -format