环境准备:
三台虚拟机 master(8)、slave1(9)、slave2(10)
centos 7.1、jdk-8u171-linux-x64.tar.gz、hadoop-2.7.3.tar.gz
0x1环境准备
首先先在三台虚拟机中创建hadoop文件夹
mdkir /usr/hadoop
在master中将hadoop解压到master的Hadoop文件夹中
tar -zxvf hadoop-2.7.3.tar.gz -C /usr/hadoop/
0x2编辑配置文件
修改配置文件 vi /usr/hadoop/hadoop-2.7.3/etc/hadoop/hadoop-env.sh
添加
export JAVA_HOME=/usr/java/jdk1.8.0_171
修改配置文件vi core-site.xml
<configuration>
<property>
<name>fs.default.name</name>
<value>http://master:9000</value>
</property>
<property>
<name>hadoop.mp.dir</name>
<value>/usr/hadoop/hadoop-2.7.3/hdfs/tmp</value>
</property>
<property>
<name>io.file.buffer.size</name>
<value>131072</value>
</property>
<property>
<name>fs.checkpoint.period</name>
<value>60</value>
</property>
<property>
<name>fs.checkpoint.size</name>
<value>67108864</value>
</property>
</configuration>
修改配置文件vi yarn-site.xml
扫描二维码关注公众号,回复:
4301479 查看本文章
<configuration>
<!-- Site specific YARN configuration properties -->
<property>
<name>yarn.resourcemanager.address</name>
<value>masetr:18040</value>
</property>
<property>
<name>yarn.resourcemanager.scheduler.address</name>
<value>masetr:18030</value>
</property>
<property>
<name>yarn.resourcemanager.webapp.address</name>
<value>masetr:18088</value>
</property>
<property>
<name>yarn.resourcemanager.resource-tracker.address</name>
<value>masetr:18025</value>
</property>
<property>
<name>yarn.resourcemanager.admin.address</name>
<value>masetr:18141</value>
</property>
<property>
<name>yarn.resourcemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.auxservices.mapreduce.shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandeler</value>
</property>
</configuration>
修改配置文件 vi slaves
slave1
slave2
修改配置文件master
master
修改配置文件hdfs-site.xml
<configuration>
<property>
<name>dfs.replication</name>
<value>2</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>file:/usr/hadoop/hadoop-2.7.3/hdfs/name</value>
<final>true</final>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>file:/usr/hadoop/hadoop-2.7.3/hdfs/data</value>
<final>true</final>
</property>
<property>
<name>dfs.namenode.secondary.http-address</name>
<value>slave1:9001</value>
</property>
<property>
<name>dfs.webhdfs.enabled</name>
<value>true</value>
</property>
<property>
<name>dfs.permissions</name>
<value>false</value>
</property>
</configuration>
创建配置文件mapred-site.xml
cp -rvf mapred-site.xml.template mapred-site.xml
然后编辑
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>
分发hadoop
scp -r /usr/hadoop/ slave1:/usr/
修改环境变量
vi /etc/profile
export HADOOP_HOME=/usr/hadoop/hadoop-2.7.3/
export PATH=$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$PATH
然后在master格式化hadoop
hadoop namenode -format
然后启动
/usr/hadoop/hadoop-2.7.3/sbin/start-all.sh