Project scene:
In the Flink code developed using Scala , a case class is defined in the method
E.g:
def csvFile(env:ExecutionEnvironment) = {
// 导入隐士转换
import org.apache.flink.api.scala._
val filePath = "file:///data/word.csv"
case class MyCaseClass(name:String, age:Int)
// case class 方式
env.readCsvFile[MyCaseClass](filePath,ignoreFirstLine = true, includedFields = Array(0,1)).print()
}
Problem Description:
An error is reported when running the code:
Exception in thread "main" java.lang.InternalError: Malformed class name
Cause Analysis:
I don't know what the reason is. I checked on Google and found a little trace. It seems to be a bug of the jdk1.8 version. It has been recorded in the scala bug submission list. After looking at it for a long time, I didn't find out how to solve it.
Scala code bug Issues have been closed in 2012. . . .
solution:
As above, when we define case classes in Flink, in order to avoid the above bugs, do not define case classes in methods .
Correct code:
object app{
def main(args: Array[String]): Unit = {
// step 1 创建环境
val env = ExecutionEnvironment.getExecutionEnvironment
csvFile(env)
}
def csvFile(env:ExecutionEnvironment) = {
// 导入隐士转换
import org.apache.flink.api.scala._
val filePath = "file:///data/word.csv"
// case class 方式
env.readCsvFile[MyCaseClass](filePath,ignoreFirstLine = true, includedFields = Array(0,1)).print()}
}
case class MyCaseClass(name:String, age:Int)