本文参照 hadoop2.2在window7上搭建单节点环境进行实施
想了解hadoop,所以就下了最新的文件,搭建相关的环境,以下为自己在win7上搭建hadoop的相关步骤。要在64位windows7中进行,否则需要自己编译hadoop-commin中的动态库和可执行文件。
1、下载hadoop-2.4.0.tar.gz,用winRAR解压。(路径不能有空格)解压到d:\hadoop目录下
2、安装java,此处就不做讲解了。jdk版本在1.7+
3、配置hadoop的环境变量HADOOP_HOME="HADOOP的安装目录" 如d:\hadoop\hadoop.2.4.0
4、在环境变量的PATH中加入HADOOP的安装目录/bin 如%HADOOP_HOME%/bin
5、配置JAVA_HOME变量为当前java的安装路径。
6、下载hadoop-commin-2.2.zip(由于没找到2.4对应的编译版本。采用2.2版本的也可以用。),因为在windows下运行,压缩包里面缺少 winutils.exe, hadoop.dll等文件,下载完成后,将要报下面的bin目录下的所有文件全部拷贝到hadoop目录下的bin文件夹下。
7、修改配置文件,以下四个文件全部在hadoop目录下的etc/hadoop目录下
修改core-site.xml,如下:
01 |
<? xml version = "1.0" encoding = "UTF-8" ?> |
02 |
<? xml-stylesheet type = "text/xsl" href = "configuration.xsl" ?> |
03 |
<!-- |
04 |
Licensed under the Apache License, Version 2.0 (the "License"); |
05 |
you may not use this file except in compliance with the License. |
06 |
You may obtain a copy of the License at |
07 |
|
09 |
|
10 |
Unless required by applicable law or agreed to in writing, software |
11 |
distributed under the License is distributed on an "AS IS" BASIS, |
12 |
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
13 |
See the License for the specific language governing permissions and |
14 |
limitations under the License. See accompanying LICENSE file. |
15 |
--> |
16 |
|
17 |
<!-- Put site-specific property overrides in this file. --> |
18 |
|
19 |
< configuration > |
20 |
< property > |
21 |
< name >fs.defaultFS</ name > |
22 |
< value >hdfs://localhost:9000</ value > |
23 |
</ property > |
24 |
</ configuration > |
修改hdfs-site.xml如下:
01 |
<? xml version = "1.0" encoding = "UTF-8" ?> |
02 |
<? xml-stylesheet type = "text/xsl" href = "configuration.xsl" ?> |
03 |
<!-- |
04 |
Licensed under the Apache License, Version 2.0 (the "License"); |
05 |
you may not use this file except in compliance with the License. |
06 |
You may obtain a copy of the License at |
07 |
|
09 |
|
10 |
Unless required by applicable law or agreed to in writing, software |
11 |
distributed under the License is distributed on an "AS IS" BASIS, |
12 |
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
13 |
See the License for the specific language governing permissions and |
14 |
limitations under the License. See accompanying LICENSE file. |
15 |
--> |
16 |
|
17 |
<!-- Put site-specific property overrides in this file. --> |
18 |
|
19 |
< configuration > |
20 |
< property > |
21 |
< name >dfs.replication</ name > |
22 |
< value >1</ value > |
23 |
</ property > |
24 |
< property > |
25 |
< name >dfs.namenode.name.dir</ name > |
26 |
< value >file:/hadoop/data/dfs/namenode</ value > |
27 |
</ property > |
28 |
< property > |
29 |
< name >dfs.datanode.data.dir</ name > |
30 |
< value >file:/hadoop/data/dfs/datanode</ value > |
31 |
</ property > |
32 |
</ configuration > |
修改yarn-site.xml如下:
01 |
<? xml version = "1.0" ?> |
02 |
<!-- |
03 |
Licensed under the Apache License, Version 2.0 (the "License"); |
04 |
you may not use this file except in compliance with the License. |
05 |
You may obtain a copy of the License at |
06 |
|
08 |
|
09 |
Unless required by applicable law or agreed to in writing, software |
10 |
distributed under the License is distributed on an "AS IS" BASIS, |
11 |
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
12 |
See the License for the specific language governing permissions and |
13 |
limitations under the License. See accompanying LICENSE file. |
14 |
--> |
15 |
< configuration > |
16 |
< property > |
17 |
< name >yarn.nodemanager.aux-services</ name > |
18 |
< value >mapreduce_shuffle</ value > |
19 |
</ property > |
20 |
< property > |
21 |
< name >yarn.nodemanager.aux-services.mapreduce.shuffle.class</ name > |
22 |
< value >org.apache.hadoop.mapred.ShuffleHandler</ value > |
23 |
</ property > |
24 |
</ configuration > |
修改mapred-site.xml如下:
01 |
<? xml version = "1.0" ?> |
02 |
<? xml-stylesheet type = "text/xsl" href = "configuration.xsl" ?> |
03 |
<!-- |
04 |
Licensed under the Apache License, Version 2.0 (the "License"); |
05 |
you may not use this file except in compliance with the License. |
06 |
You may obtain a copy of the License at |
07 |
|
09 |
|
10 |
Unless required by applicable law or agreed to in writing, software |
11 |
distributed under the License is distributed on an "AS IS" BASIS, |
12 |
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
13 |
See the License for the specific language governing permissions and |
14 |
limitations under the License. See accompanying LICENSE file. |
15 |
--> |
16 |
|
17 |
<!-- Put site-specific property overrides in this file. --> |
18 |
|
19 |
< configuration > |
20 |
< property > |
21 |
< name >mapreduce.framework.name</ name > |
22 |
< value >yarn</ value > |
23 |
</ property > |
24 |
</ configuration > |
然后打开cmd,运行hadoop namenode -format命令,运行结果基本如下:
d:\hadoop\hadoop-2.4.0\bin>hdfs namenode -format
注意观查控制台信息如果没有报错会在/hadoop/data/dfs/namenode下和/hadoop/data/dfs/datanode下生成一些文件。
然后在cmd下切换目录到hadoop2.4.0目录下的sbin目录下,运行start-all 会打开四个cmd窗口,可以打开浏览器输入 http://localhost:8042以及http://localhost:50070查看是否配置成功!