Spark踩坑：JsonMappingException: Incompatible Jackson version: 2.9.6

版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。

本文链接： https://blog.csdn.net/weixin_41876523/article/details/81740953

在Spark2中，如果使用了kafka库，则很容易产生如下错误：

Exception in thread "main" java.lang.ExceptionInInitializerError
    at org.apache.spark.streaming.dstream.InputDStream.<init>(InputDStream.scala:80)
    at org.apache.spark.streaming.dstream.ReceiverInputDStream.<init>(ReceiverInputDStream.scala:42)
    at org.apache.spark.streaming.kafka.KafkaInputDStream.<init>(KafkaInputDStream.scala:56)
    at org.apache.spark.streaming.kafka.KafkaUtils$.createStream(KafkaUtils.scala:92)
    at AppMain$.delayedEndpoint$AppMain$1(AppMain.scala:29)
    at AppMain$delayedInit$body.apply(AppMain.scala:12)
    at scala.Function0$class.apply$mcV$sp(Function0.scala:34)
    at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
    at scala.App$$anonfun$main$1.apply(App.scala:76)
	at scala.App$$anonfun$main$1.apply(App.scala:76)
    at scala.collection.immutable.List.foreach(List.scala:392)
    at scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:35)
    at scala.App$class.main(App.scala:76)
    at AppMain$.main(AppMain.scala:12)
    at AppMain.main(AppMain.scala)
Caused by: com.fasterxml.jackson.databind.JsonMappingException: Incompatible Jackson version: 2.9.6
    at com.fasterxml.jackson.module.scala.JacksonModule$class.setupModule(JacksonModule.scala:64)
    at com.fasterxml.jackson.module.scala.DefaultScalaModule.setupModule(DefaultScalaModule.scala:19)
    at com.fasterxml.jackson.databind.ObjectMapper.registerModule(ObjectMapper.java:751)
    at org.apache.spark.rdd.RDDOperationScope$.<init>(RDDOperationScope.scala:82)
    at org.apache.spark.rdd.RDDOperationScope$.<clinit>(RDDOperationScope.scala)
    ... 15 more

• 这是由于Jackson这个工具库的版本不一致导致的。解决方案：首先在Kafka的依赖项中，排除对于Jackon的依赖，从而阻止Maven自动导入高版本的库，随后手动添加较低版本Jackon库的依赖项，重新import即可。

添加排除项：

<dependency>
    <groupId> org.apache.kafka</groupId>
    <artifactId>kafka_2.11</artifactId>
    <version>1.1.1</version>
    <exclusions>
        <exclusion>
            <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>*</artifactId>
        </exclusion>
    </exclusions>
</dependency>

• 添加依赖项：

<dependency>
    <groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-core</artifactId>
    <version>2.6.6</version>
</dependency>

• 重新import，可见问题解决。