win7系统进行hadoop 2.3 源代码编译

1、从官网上下载源代码,并解压;
2、下载maven2.3.1版本(2.0.3版本也执行过没问题)
3、到%hadoop-2.3.0-src%/hadoop-maven-plugins/目录下, 执行
mvn clean install
4、在有些公司或网络下,可能mvn install命令会执行失败,报maven的资源库无法访问,我碰到的是会自动连到report1那个上面,这个可以修改%MAVEN_HOME%/conf/settings.xml,修改<mirrors></mirrors>中的内容,如下:

  <mirrors>
    <mirror>
      <id>mirrorId</id>
      <mirrorOf>repositoryId</mirrorOf>
      <name>Human Readable Name for this Mirror.</name>
      <url>http://repo.maven.apache.org/maven2/</url>//这个url如果不可用,可以使用其它能用的替换
    </mirror>
  </mirrors>
5、如果有的公司上网是通过proxy,则需要继续修改同一个文件的<proxies></proxies>,例如:
  <proxies>
    <proxy>
      <id>optional</id>
      <active>true</active>
      <protocol>http</protocol>
      <username>proxyuser</username>
      <password>proxypass</password>
      <host>proxy.host.net</host>
      <port>80</port>
      <nonProxyHosts>local.net|some.host.com</nonProxyHosts>
    </proxy>
  </proxies>
6、到这里,基本上mvn clean install这个命令可以执行完了;然后回退到上一个目录,也就是cd %hadoop-2.3.0-src%/,执行:
mvn eclipse:eclipse -DskipTests
也即是跳过测试了
7、上面命令执行过程中,一般会出现一个问题:
[ERROR] Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:2.3.0:protoc (compile-protoc) on project hadoop-common: org.apache.maven.plugin.MojoExecutionException: 'protoc --version' did not return a version -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR][ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command

[ERROR]   mvn <goals> -rf :hadoop-common

8、上面这个错误,可以看出是由于没有找到proto引起的:org.apache.maven.plugin.MojoExecutionException: 'protoc --version' did not return a version -> [Help 1]
9、win7下,可以直接从网上下载protoc-2.5.0-win32.zip,win7 64位也可以执行,我用的就是64位操作系统。然后设置环境变量,进行加载即可,也可以用一个偷懒的方法:
把protoc.exe编译器,放入program files/protoc/自定的文件夹下,将这个目录追加入path环境变量。这样mvn就能调用protoc.exe,并且你自己也可以在cmd中随意执行。
10、重新执行mvn eclipse:eclipse -DskipTests
11、等到的最终执行完的结果:
[INFO] Apache Hadoop Main ................................ SUCCESS [  1.222 s]
[INFO] Apache Hadoop Project POM ......................... SUCCESS [  1.376 s]
[INFO] Apache Hadoop Annotations ......................... SUCCESS [  0.263 s]
[INFO] Apache Hadoop Project Dist POM .................... SUCCESS [  0.189 s]
[INFO] Apache Hadoop Assemblies .......................... SUCCESS [  0.214 s]
[INFO] Apache Hadoop Maven Plugins ....................... SUCCESS [  2.718 s]
[INFO] Apache Hadoop MiniKDC ............................. SUCCESS [  2.666 s]
[INFO] Apache Hadoop Auth ................................ SUCCESS [  1.948 s]
[INFO] Apache Hadoop Auth Examples ....................... SUCCESS [  0.191 s]
[INFO] Apache Hadoop Common .............................. SUCCESS [  5.259 s]
[INFO] Apache Hadoop NFS ................................. SUCCESS [01:15 min]
[INFO] Apache Hadoop Common Project ...................... SUCCESS [  0.132 s]
[INFO] Apache Hadoop HDFS ................................ SUCCESS [01:55 min]
[INFO] Apache Hadoop HttpFS .............................. SUCCESS [ 25.091 s]
[INFO] Apache Hadoop HDFS BookKeeper Journal ............. SUCCESS [ 15.946 s]
[INFO] Apache Hadoop HDFS-NFS ............................ SUCCESS [  4.452 s]
[INFO] Apache Hadoop HDFS Project ........................ SUCCESS [  0.497 s]
[INFO] hadoop-yarn ....................................... SUCCESS [  0.150 s]
[INFO] hadoop-yarn-api ................................... SUCCESS [  2.251 s]
[INFO] hadoop-yarn-common ................................ SUCCESS [ 26.932 s]
[INFO] hadoop-yarn-server ................................ SUCCESS [  0.143 s]
[INFO] hadoop-yarn-server-common ......................... SUCCESS [  8.565 s]
[INFO] hadoop-yarn-server-nodemanager .................... SUCCESS [ 51.856 s]
[INFO] hadoop-yarn-server-web-proxy ...................... SUCCESS [  2.283 s]
[INFO] hadoop-yarn-server-resourcemanager ................ SUCCESS [  4.427 s]
[INFO] hadoop-yarn-server-tests .......................... SUCCESS [  9.880 s]
[INFO] hadoop-yarn-client ................................ SUCCESS [  2.852 s]
[INFO] hadoop-yarn-applications .......................... SUCCESS [  0.112 s]
[INFO] hadoop-yarn-applications-distributedshell ......... SUCCESS [  3.241 s]
[INFO] hadoop-yarn-applications-unmanaged-am-launcher .... SUCCESS [  1.021 s]
[INFO] hadoop-yarn-site .................................. SUCCESS [  0.099 s]
[INFO] hadoop-yarn-project ............................... SUCCESS [  0.151 s]
[INFO] hadoop-mapreduce-client ........................... SUCCESS [  0.263 s]
[INFO] hadoop-mapreduce-client-core ...................... SUCCESS [ 25.972 s]
[INFO] hadoop-mapreduce-client-common .................... SUCCESS [  8.810 s]
[INFO] hadoop-mapreduce-client-shuffle ................... SUCCESS [  1.110 s]
[INFO] hadoop-mapreduce-client-app ....................... SUCCESS [  6.729 s]
[INFO] hadoop-mapreduce-client-hs ........................ SUCCESS [  8.809 s]
[INFO] hadoop-mapreduce-client-jobclient ................. SUCCESS [ 10.474 s]
[INFO] hadoop-mapreduce-client-hs-plugins ................ SUCCESS [  0.708 s]
[INFO] Apache Hadoop MapReduce Examples .................. SUCCESS [ 18.896 s]
[INFO] hadoop-mapreduce .................................. SUCCESS [  0.199 s]
[INFO] Apache Hadoop MapReduce Streaming ................. SUCCESS [  8.281 s]
[INFO] Apache Hadoop Distributed Copy .................... SUCCESS [03:25 min]
[INFO] Apache Hadoop Archives ............................ SUCCESS [  2.256 s]
[INFO] Apache Hadoop Rumen ............................... SUCCESS [  1.381 s]
[INFO] Apache Hadoop Gridmix ............................. SUCCESS [  4.527 s]
[INFO] Apache Hadoop Data Join ........................... SUCCESS [  0.865 s]
[INFO] Apache Hadoop Extras .............................. SUCCESS [  0.690 s]
[INFO] Apache Hadoop Pipes ............................... SUCCESS [  0.139 s]
[INFO] Apache Hadoop OpenStack support ................... SUCCESS [  1.876 s]
[INFO] Apache Hadoop Client .............................. SUCCESS [  2.828 s]
[INFO] Apache Hadoop Mini-Cluster ........................ SUCCESS [  1.942 s]
[INFO] Apache Hadoop Scheduler Load Simulator ............ SUCCESS [ 14.240 s]
[INFO] Apache Hadoop Tools Dist .......................... SUCCESS [  7.795 s]
[INFO] Apache Hadoop Tools ............................... SUCCESS [  0.123 s]
[INFO] Apache Hadoop Distribution ........................ SUCCESS [  0.520 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 11:45 min
[INFO] Finished at: 2014-04-05T12:52:52+08:00
[INFO] Final Memory: 79M/307M
[INFO] ------------------------------------------------------------------------
D:\Download\bigdata\hadoop-2.3.0-src>

12、最后在Eclipse中按照以下流程导入源代码:“File” →  “Import”→  “Existing Projects into Workspace”。(我用的是eclipse-java-kepler)
13,代码导入后,可能会出现一些java类不存在,例如:EchoResponseProto.java/EmptyResponseProto,主要是测试类,可以通过google解决。

猜你喜欢

转载自trampeagle.iteye.com/blog/2041360
2.3
今日推荐