Flink编译避坑操作

编译前提

  • 安装maven3.x,配置远程仓库,这里保险配置2个:
<mirror>
        <id>nexus</id>
        <mirrorOf>*</mirrorOf>
        <url>http://maven.aliyun.com/nexus/content/groups/public/</url>
</mirror>
<mirror>
        <id>nexus-public-snapshots</id>
        <mirrorOf>public-snapshots</mirrorOf>
        <url>http://maven.aliyun.com/nexus/content/repositories/snapshots/</url>
</mirror>
  • 安装jdk1.8u51以上
  • 安装git

下载编译源码

git clone https://github.com/apache/flink.git
git tag
git checkout release-1.6.2
mvn clean install -DskipTests -Dfast

注意第一次clean,后面出错了就用mvn install -DskipTests -Dfast

本次编译由于网络时间耗时大约一天,当成功编译完成后,上述几种编译方式最终都能在当前flink的code path下编译出完整的flink binary,可以在flink-dist/target/目录中看到:
在这里插入图片描述

可能出现的坑一

[ERROR] Failed to execute goal on project flink-mapr-fs: Could not resolve dependencies for project org.apache.flink:flink-mapr-fs:jar:1.6.2: Could not find artifact com.mapr.hadoop:maprfs:jar:5.2.1-mapr in nexus-osc (http://maven.aliyun.com/nexus/content/repositories/central) -> [Help 1]

如上说明你缺少一个flink-mapr-fs:jar,自己去下载,然后命令执行安装:
1.下载maprfs jar包
通过手动下载maprfs-5.2.1-mapr.jar包,下载地址地址:https://repository.mapr.com/nexus/content/groups/mapr-public/com/mapr/hadoop/maprfs/5.2.1-mapr/

2.上传到Linux

3.手动安装

mvn install:install-file -DgroupId=com.mapr.hadoop -DartifactId=maprfs -Dversion=5.2.1-mapr -Dpackaging=jar  -Dfile=maprfs-5.2.1-mapr.jar的位置

可能出现的坑二

[ERROR] /home/hadoop/opensource/flink/flink-filesystems/flink-mapr-fs/src/main/java/org/apache/flink/runtime/fs/maprfs/MapRFileSystem.java:[70,44] package org.apache.hadoop.fs does not exist

如上说明缺失org.apache.hadoop.fs包,报错找不到。解决如下:

在flink-mapr-fs模块的pom文件中添加如下依赖(hadoop.version不用你写具体的 下面代码粘贴进去就可以了):

<dependency>
	<groupId>org.apache.hadoop</groupId>
	<artifactId>hadoop-common</artifactId>
	<version>${hadoop.version}</version>
</dependency>

可能出现的坑三

[ERROR] Failed to execute goal on project flink-avro-confluent-registry: Could not resolve dependencies for project org.apache.flink:flink-avro-confluent-registry:jar:1.6.2: Could not find artifact io.confluent:kafka-schema-registry-client:jar:3.3.1 in nexus-osc (http://maven.aliyun.com/nexus/content/repositories/central) -> [Help 1]

如上告诉我们报错缺少kafka-schema-registry-client-3.3.1.jar 包,下载地址如下:
http://packages.confluent.io/maven/io/confluent/kafka-schema-registry-client/3.3.1/kafka-schema-registry-client-3.3.1.jar

上传到Linux下,执行如下命令

mvn install:install-file -DgroupId=io.confluent -DartifactId=kafka-schema-registry-client -Dversion=3.3.1 -Dpackaging=jar  -Dfile=kafka-schema-registry-client-3.3.1.jar的位置
发布了3 篇原创文章 · 获赞 86 · 访问量 22万+

猜你喜欢

转载自blog.csdn.net/qq285016127/article/details/104243110