1、环境需要：
版本：hadoop3.0.3
----------------------------------------------------------------------------------
Requirements:
* Unix System
* JDK 1.8
* Maven 3.3 or later
* ProtocolBuffer 2.5.0
* CMake 3.1 or newer (if compiling native code)
* Zlib devel (if compiling native code)
* openssl devel (if compiling native hadoop-pipes and to get the best HDFS encryption performance)
* Linux FUSE (Filesystem in Userspace) version 2.6 or above (if compiling fuse_dfs)
* Internet connection for first build (to fetch all Maven and Hadoop dependencies)
* python (for releasedocs)
* bats (for shell code testing)
* Node.js / bower / Ember-cli (for YARN UI v2 building)
2、踩过的坑
1、mvn 配置镜像库，如果不配置默认到maven的官方库，下载速度慢死。
2、源代码以下几个pom.xml内置的仓库也需要修改到镜像库，不然会有几个地方下载慢的要死。原因：构件优先在pom.xml内配置的仓库地址搜索，其次到mvn的镜像库地址。
./pom.xml ./hadoop-project/pom.xml
3、protocbuf2.5要去github下载源码编译，下载比较慢。
4、其他Requirements，可以通过yum下载安装
5、阿里云mvn站、amzone缺少DynamoDBLocal-1.11.86.jar。需要手工安装，
1登陆http://mvnrepository.com/ 这个地址搜索下载。
2、mvn install:install-file -Dfile="/root/mvntemp/DynamoDBLocal-1.11.86.jar" -DgroupId=com.amazonaws -DartifactId=DynamoDBLocal -Dversion=1.11.86 -Dpackaging=jar
6、cmake要3.1版本的，yum没有，要单独下载。
打包
$ mvn clean package -Pdist,native -DskipTests -Dtar
mvn clean package -Pdist,native -DskipTests -Dtar
7、从错误的地方开始继续打包
mvn clean package -Pdist,native -DskipTests -Dtar -rf :hadoop-pipes
3一些配置文档
<!-- https://mvnrepository.com/artifact/com.amazonaws/DynamoDBLocal -->
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>DynamoDBLocal</artifactId>
<version>1.11.86</version>
<scope>test</scope>
</dependency>
[DEBUG] com.amazonaws:DynamoDBLocal:jar:1.11.86:test
Downloading from alimaven: https://maven.aliyun.com/repository/public/com/amazonaws/DynamoDBLocal/1.11.86/DynamoDBLocal-1.11.86.jar
[DEBUG] Writing tracking file /root/.m2/repository/com/amazonaws/DynamoDBLocal/1.11.86/DynamoDBLocal-1.11.86.jar.lastUpdated
<mirror>
<id>alimaven</id>
<name>aliyun maven</name>
<url>http://maven.aliyun.com/nexus/content/groups/public/</url>
<mirrorOf>central</mirrorOf>
</mirror>
<repositories>
<repository>
<id>dynamodb-local-oregon</id>
<name>DynamoDB Local Release Repository</name>
<url>http://maven.aliyun.com/nexus/content/groups/public/</url>
</repository>
</repositories>
<distMgmtSnapshotsId>apache.snapshots.https</distMgmtSnapshotsId>
<distMgmtSnapshotsName>Apache Development Snapshot Repository</distMgmtSnapshotsName>
<distMgmtSnapshotsUrl>http://maven.aliyun.com/nexus/content/groups/public/</distMgmtSnapshotsUrl>
<distMgmtStagingId>apache.staging.https</distMgmtStagingId>
<distMgmtStagingName>Apache Release Distribution Repository</distMgmtStagingName>
<distMgmtStagingUrl>http://maven.aliyun.com/nexus/content/groups/public/</distMgmtStagingUrl>
4、打包编译日志
mvn clean package -Pdist,native -DskipTests –Dtar
目标目录生成打包文件，
[root@localhost target]# pwd
/root/hdsource/hadoop-3.0.3-src/hadoop-dist/target
[root@localhost target]# ls -ltr
总用量 269380
drwxr-xr-x. 2 root root 4096 8月 18 08:16 antrun
drwxr-xr-x. 2 root root 4096 8月 18 08:16 test-dir
drwxr-xr-x. 3 root root 4096 8月 18 08:16 maven-shared-archive-resources
drwxr-xr-x. 3 root root 4096 8月 18 08:16 classes
drwxr-xr-x. 3 root root 4096 8月 18 08:16 test-classes
drwxr-xr-x. 9 root root 4096 8月 18 08:16 hadoop-3.0.3
-rw-r--r--. 1 root root 275815889 8月 18 08:16 hadoop-3.0.3.tar.gz
[root@localhost target]#
打包编译日志
------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Hadoop Main 3.0.3 ........................... SUCCESS [ 3.927 s]
[INFO] Apache Hadoop Build Tools .......................... SUCCESS [ 15.215 s]
[INFO] Apache Hadoop Project POM .......................... SUCCESS [ 2.990 s]
[INFO] Apache Hadoop Annotations .......................... SUCCESS [ 8.739 s]
[INFO] Apache Hadoop Assemblies ........................... SUCCESS [ 0.500 s]
[INFO] Apache Hadoop Project Dist POM ..................... SUCCESS [ 4.316 s]
[INFO] Apache Hadoop Maven Plugins ........................ SUCCESS [ 11.936 s]
[INFO] Apache Hadoop MiniKDC .............................. SUCCESS [ 5.442 s]
[INFO] Apache Hadoop Auth ................................. SUCCESS [ 19.166 s]
[INFO] Apache Hadoop Auth Examples ........................ SUCCESS [ 7.160 s]
[INFO] Apache Hadoop Common ............................... SUCCESS [03:33 min]
[INFO] Apache Hadoop NFS .................................. SUCCESS [ 19.374 s]
[INFO] Apache Hadoop KMS .................................. SUCCESS [ 14.788 s]
[INFO] Apache Hadoop Common Project ....................... SUCCESS [ 0.194 s]
[INFO] Apache Hadoop HDFS Client .......................... SUCCESS [01:54 min]
[INFO] Apache Hadoop HDFS ................................. SUCCESS [04:27 min]
[INFO] Apache Hadoop HDFS Native Client ................... SUCCESS [ 15.106 s]
[INFO] Apache Hadoop HttpFS ............................... SUCCESS [ 23.684 s]
[INFO] Apache Hadoop HDFS-NFS ............................. SUCCESS [ 9.426 s]
[INFO] Apache Hadoop HDFS-RBF ............................. SUCCESS [ 50.275 s]
[INFO] Apache Hadoop HDFS Project ......................... SUCCESS [ 0.462 s]
[INFO] Apache Hadoop YARN ................................. SUCCESS [ 0.251 s]
[INFO] Apache Hadoop YARN API ............................. SUCCESS [ 43.917 s]
[INFO] Apache Hadoop YARN Common .......................... SUCCESS [01:58 min]
[INFO] Apache Hadoop YARN Server .......................... SUCCESS [ 0.859 s]
[INFO] Apache Hadoop YARN Server Common ................... SUCCESS [ 44.076 s]
[INFO] Apache Hadoop YARN Registry ........................ SUCCESS [ 13.566 s]
[INFO] Apache Hadoop YARN NodeManager ..................... SUCCESS [01:40 min]
[INFO] Apache Hadoop YARN Web Proxy ....................... SUCCESS [ 15.754 s]
[INFO] Apache Hadoop YARN ApplicationHistoryService ....... SUCCESS [ 19.548 s]
[INFO] Apache Hadoop YARN Timeline Service ................ SUCCESS [ 12.406 s]
[INFO] Apache Hadoop YARN ResourceManager ................. SUCCESS [01:10 min]
[INFO] Apache Hadoop YARN Server Tests .................... SUCCESS [ 12.081 s]
[INFO] Apache Hadoop YARN Client .......................... SUCCESS [ 37.168 s]
[INFO] Apache Hadoop YARN SharedCacheManager .............. SUCCESS [ 12.077 s]
[INFO] Apache Hadoop YARN Timeline Plugin Storage ......... SUCCESS [ 9.840 s]
[INFO] Apache Hadoop YARN TimelineService HBase Backend ... SUCCESS [ 33.311 s]
[INFO] Apache Hadoop YARN Timeline Service HBase tests .... SUCCESS [ 9.911 s]
[INFO] Apache Hadoop YARN Router .......................... SUCCESS [ 15.635 s]
[INFO] Apache Hadoop YARN Applications .................... SUCCESS [ 0.113 s]
[INFO] Apache Hadoop YARN DistributedShell ................ SUCCESS [ 9.116 s]
[INFO] Apache Hadoop YARN Unmanaged Am Launcher ........... SUCCESS [ 5.872 s]
[INFO] Apache Hadoop YARN Site ............................ SUCCESS [ 0.124 s]
[INFO] Apache Hadoop YARN UI .............................. SUCCESS [ 0.360 s]
[INFO] Apache Hadoop YARN Project ......................... SUCCESS [ 23.455 s]
[INFO] Apache Hadoop MapReduce Client ..................... SUCCESS [ 1.678 s]
[INFO] Apache Hadoop MapReduce Core ....................... SUCCESS [01:01 min]
[INFO] Apache Hadoop MapReduce Common ..................... SUCCESS [ 41.308 s]
[INFO] Apache Hadoop MapReduce Shuffle .................... SUCCESS [ 15.860 s]
[INFO] Apache Hadoop MapReduce App ........................ SUCCESS [ 23.159 s]
[INFO] Apache Hadoop MapReduce HistoryServer .............. SUCCESS [ 14.141 s]
[INFO] Apache Hadoop MapReduce JobClient .................. SUCCESS [ 18.734 s]
[INFO] Apache Hadoop MapReduce HistoryServer Plugins ...... SUCCESS [ 5.382 s]
[INFO] Apache Hadoop MapReduce NativeTask ................. SUCCESS [01:59 min]
[INFO] Apache Hadoop MapReduce Examples ................... SUCCESS [ 13.407 s]
[INFO] Apache Hadoop MapReduce ............................ SUCCESS [ 9.071 s]
[INFO] Apache Hadoop MapReduce Streaming .................. SUCCESS [ 10.556 s]
[INFO] Apache Hadoop Distributed Copy ..................... SUCCESS [ 14.021 s]
[INFO] Apache Hadoop Archives ............................. SUCCESS [ 5.416 s]
[INFO] Apache Hadoop Archive Logs ......................... SUCCESS [ 5.224 s]
[INFO] Apache Hadoop Rumen ................................ SUCCESS [ 11.460 s]
[INFO] Apache Hadoop Gridmix .............................. SUCCESS [ 9.583 s]
[INFO] Apache Hadoop Data Join ............................ SUCCESS [ 5.858 s]
[INFO] Apache Hadoop Extras ............................... SUCCESS [ 5.076 s]
[INFO] Apache Hadoop Pipes ................................ SUCCESS [ 13.922 s]
[INFO] Apache Hadoop OpenStack support .................... SUCCESS [ 12.983 s]
[INFO] Apache Hadoop Amazon Web Services support .......... SUCCESS [ 22.870 s]
[INFO] Apache Hadoop Kafka Library support ................ SUCCESS [ 9.432 s]
[INFO] Apache Hadoop Azure support ........................ SUCCESS [ 15.243 s]
[INFO] Apache Hadoop Aliyun OSS support ................... SUCCESS [ 11.155 s]
[INFO] Apache Hadoop Client Aggregator .................... SUCCESS [ 6.909 s]
[INFO] Apache Hadoop Mini-Cluster ......................... SUCCESS [ 7.115 s]
[INFO] Apache Hadoop Scheduler Load Simulator ............. SUCCESS [ 12.102 s]
[INFO] Apache Hadoop Resource Estimator Service ........... SUCCESS [ 11.346 s]
[INFO] Apache Hadoop Azure Data Lake support .............. SUCCESS [ 7.746 s]
[INFO] Apache Hadoop Tools Dist ........................... SUCCESS [ 26.061 s]
[INFO] Apache Hadoop Tools ................................ SUCCESS [ 0.141 s]
[INFO] Apache Hadoop Client API ........................... SUCCESS [02:32 min]
[INFO] Apache Hadoop Client Runtime ....................... SUCCESS [01:50 min]
[INFO] Apache Hadoop Client Packaging Invariants .......... SUCCESS [ 0.848 s]
[INFO] Apache Hadoop Client Test Minicluster .............. SUCCESS [02:29 min]
[INFO] Apache Hadoop Client Packaging Invariants for Test . SUCCESS [ 0.411 s]
[INFO] Apache Hadoop Client Packaging Integration Tests ... SUCCESS [ 0.331 s]
[INFO] Apache Hadoop Distribution ......................... SUCCESS [01:07 min]
[INFO] Apache Hadoop Client Modules ....................... SUCCESS [ 0.223 s]
[INFO] Apache Hadoop Cloud Storage ........................ SUCCESS [ 4.106 s]
[INFO] Apache Hadoop Cloud Storage Project 3.0.3 .......... SUCCESS [ 0.101 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 40:57 min
[INFO] Finished at: 2018-08-18T08:17:28+08:00
[INFO] ------------------------------------------------------------------------
[root@localhost hadoop-3.0.3-src]#