环境要求：

jdk8

maven3.5.4+

git bash

编译准备

给本地的maven settings加上一些仓库的mirror, 加速编译过程中依赖的下载

 <mirrors>
	<mirror>
	    <id>alimaven</id>
	    <name>aliyun maven</name>
	    <url>http://maven.aliyun.com/nexus/content/groups/public/</url>
	    <mirrorOf>central</mirrorOf>
	</mirror>
	<mirror>
    	<id>ibiblio</id>
    	<name>Mirror from Maven ibiblio</name>
    	<url>http://mirrors.ibiblio.org/pub/mirrors/maven2/</url>
    	<mirrorOf>central</mirrorOf>
	</mirror>
	<mirror>
		<id>sprintio</id>
		<mirrorOf>central</mirrorOf>
		<name>Human Readable Name for this Mirror.</name>
		<url>https://repo.spring.io/libs-snapshot/</url>
	</mirror>
	<mirror>
		<id>central</id>
		<name>Maven Repository Switchboard</name>
		<url>http://repo1.maven.org/maven2/</url>
		<mirrorOf>central</mirrorOf>
    </mirror>
	<mirror>
	    <id>central</id>
	    <name>Maven Repository Switchboard</name>
	    <url>http://repo1.maven.apache.org/maven2/</url>
	    <mirrorOf>central</mirrorOf>
	</mirror>
	<mirror>
		<id>repo2</id>
		<mirrorOf>central</mirrorOf>
		<name>Human Readable Name for this Mirror.</name>
		<url>http://repo2.maven.org/maven2/</url>
    </mirror> 
</mirrors>

编辑flume源码项目的pom

将repositories替换成aliyun的

<repositories><!-- 代码库 -->  
      <repository>  
          <id>maven-ali</id>  
          <url>http://maven.aliyun.com/nexus/content/groups/public/</url>  
          <releases>  
              <enabled>true</enabled>  
          </releases>  
          <snapshots>  
              <enabled>true</enabled>  
              <updatePolicy>always</updatePolicy>  
              <checksumPolicy>fail</checksumPolicy>  
          </snapshots>  
      </repository>  
  </repositories>

编辑子模块flume-ng-sinks的pom文件

M!-- 下面2个注起来的子模块会导致编译失败，而且去掉他们不影响我们在大数据环境下的常规使用-->
 <modules>
    <module>flume-hdfs-sink</module>
<!--    <module>flume-irc-sink</module>-->
    <module>flume-ng-hbase-sink</module>
    <module>flume-ng-hbase2-sink</module>
    <module>flume-ng-elasticsearch-sink</module>
<!--    <module>flume-ng-morphline-solr-sink</module>-->
    <module>flume-ng-kafka-sink</module>
    <module>flume-http-sink</module>
    <module>flume-dataset-sink</module>
    <module>flume-hive-sink</module>
  </modules>

开始编译

在flume的源码工程的根目录下，打开git bash，先查看下maven参数信息

$ mvn -v
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: D:\apache-maven-3.6.3
Java version: 1.8.0_92, vendor: Oracle Corporation, runtime: D:\Developers\jdk1.8.0_92\jre
Default locale: zh_CN, platform encoding: GBK
OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"

我们发现maven的编码为GBK，而在生产服务器上的编码一般为UTF-8. 所以我们需要更改

在编译项目时，需要设置一定的堆内存，通过命令设置maven的编码和执行参数

扫描二维码关注公众号，回复： 11500946 查看本文章

export MAVEN_OPTS=" -Xms1g -Xmx1g -XX:PermSize=256m -XX:MaxPermSize=512m -Dfile.encoding=UTF-8"

再查看maven参数

$ mvn -v
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: D:\apache-maven-3.6.3
Java version: 1.8.0_92, vendor: Oracle Corporation, runtime: D:\Developers\jdk1.8.0_92\jre
Default locale: zh_CN, platform encoding: UTF-8
OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"

好了，当前编译会话窗口的编码已经是UTF-8了。

#执行maven清理
mvn clean

#编译打包
mvn package -DskipTests

一段时间后，出现如下信息

[INFO] Reactor Summary for Apache Flume 1.9.0:
[INFO]
[INFO] Build Support ...................................... SUCCESS [  2.542 s]
[INFO] Apache Flume ....................................... SUCCESS [  3.595 s]
[INFO] Flume NG SDK ....................................... SUCCESS [  9.207 s]
[INFO] Flume NG Hadoop Credential Store Config Filter ..... SUCCESS [  0.054 s]
[INFO] Flume NG Config Filters API ........................ SUCCESS [  0.335 s]
[INFO] Flume NG Configuration ............................. SUCCESS [  2.419 s]
[INFO] Flume Auth ......................................... SUCCESS [  5.857 s]
[INFO] Flume NG Core ...................................... SUCCESS [ 19.793 s]
[INFO] Flume NG Sinks ..................................... SUCCESS [  0.100 s]
[INFO] Flume NG HDFS Sink ................................. SUCCESS [  8.497 s]
[INFO] Flume NG Channels .................................. SUCCESS [  0.088 s]
[INFO] Flume NG JDBC channel .............................. SUCCESS [  2.903 s]
[INFO] Flume NG file-based channel ........................ SUCCESS [  5.314 s]
[INFO] Flume NG Spillable Memory channel .................. SUCCESS [  1.877 s]
[INFO] Flume NG Node ...................................... SUCCESS [  7.456 s]
[INFO] Flume NG Embedded Agent ............................ SUCCESS [  2.202 s]
[INFO] Flume NG HBase Sink ................................ SUCCESS [  7.854 s]
[INFO] Flume NG HBase2 Sink ............................... SUCCESS [ 44.701 s]
[INFO] Flume NG ElasticSearch Sink ........................ SUCCESS [  4.038 s]
[INFO] Flume Shared Utils ................................. SUCCESS [  0.046 s]
[INFO] Flume Shared Kafka ................................. SUCCESS [  1.065 s]
[INFO] Flume Shared Kafka Test Utils ...................... SUCCESS [  1.517 s]
[INFO] Flume Kafka Sink ................................... SUCCESS [  2.920 s]
[INFO] Flume HTTP/S Sink .................................. SUCCESS [  1.848 s]
[INFO] Flume NG Kite Dataset Sink ......................... SUCCESS [  7.636 s]
[INFO] Flume NG Hive Sink ................................. SUCCESS [  5.678 s]
[INFO] Flume Sources ...................................... SUCCESS [  0.040 s]
[INFO] Flume Scribe Source ................................ SUCCESS [  2.127 s]
[INFO] Flume JMS Source ................................... SUCCESS [  2.923 s]
[INFO] Flume Twitter Source ............................... SUCCESS [  2.094 s]
[INFO] Flume Kafka Source ................................. SUCCESS [  2.941 s]
[INFO] Flume Taildir Source ............................... SUCCESS [  2.067 s]
[INFO] flume-kafka-channel ................................ SUCCESS [  2.648 s]
[INFO] Flume legacy Sources ............................... SUCCESS [  0.037 s]
[INFO] Flume legacy Avro source ........................... SUCCESS [  2.343 s]
[INFO] Flume legacy Thrift Source ......................... SUCCESS [  2.166 s]
[INFO] Flume NG Environment Variable Config Filter ........ SUCCESS [  0.646 s]
[INFO] flume-ng-hadoop-credential-store-config-filter ..... SUCCESS [  2.056 s]
[INFO] Flume NG External Process Config Filter ............ SUCCESS [  0.627 s]
[INFO] Flume NG Clients ................................... SUCCESS [  0.044 s]
[INFO] Flume NG Log4j Appender ............................ SUCCESS [  9.799 s]
[INFO] Flume NG Tools ..................................... SUCCESS [  1.863 s]
[INFO] Flume NG distribution .............................. SUCCESS [ 38.532 s]
[INFO] Flume NG Integration Tests ......................... SUCCESS [  3.600 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  03:47 min
[INFO] Finished at: 2020-05-22T15:07:16+08:00
[INFO] --------------------------------------------------------

编译成功！

注：我的编译时间只有3min47s，这是因为我编译了多次，首次编译要比这耗时的多

Win10环境下编译Flume源码

环境要求：

编译准备

开始编译

猜你喜欢