Flink executes jar error: java.io.IOException: Error opening the Input Split file or java.io.FileNotFoundException

1. Error content

When Flink executes the jar, the following error is reported:

org.apache.flink.client.program.ProgramInvocationException: Job failed. (JobID: 80bbed7b2180c6fd5904c29e7e8af447)
	at org.apache.flink.client.program.rest.RestClusterClient.submitJob(RestClusterClient.java:268)
	at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:486)
	at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:474)
	at org.apache.flink.client.program.ContextEnvironment.execute(ContextEnvironment.java:62)
	at org.apache.flink.api.java.ExecutionEnvironment.execute(ExecutionEnvironment.java:816)
	at org.apache.flink.api.java.DataSet.collect(DataSet.java:413)
	at org.apache.flink.api.java.DataSet.print(DataSet.java:1652)
	at org.apache.flink.examples.java.wordcount.WordCount.main(WordCount.java:88)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:529)
	at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:421)
	at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:426)
	at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:816)
	at org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:290)
	at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:216)
	at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1053)
	at org.apache.flink.client.cli.CliFrontend.lambda$main$11(CliFrontend.java:1129)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836)
	at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
	at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1129)
Caused by: org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
	at org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:146)
	at org.apache.flink.client.program.rest.RestClusterClient.submitJob(RestClusterClient.java:265)
	... 24 more
Caused by: java.io.IOException: Error opening the Input Split file:/opt/module/datas/word.txt [0,64]: /opt/module/datas/word.txt (没有那个文件或目录)
	at org.apache.flink.api.common.io.FileInputFormat.open(FileInputFormat.java:824)
	at org.apache.flink.api.common.io.DelimitedInputFormat.open(DelimitedInputFormat.java:470)
	at org.apache.flink.api.common.io.DelimitedInputFormat.open(DelimitedInputFormat.java:47)
	at org.apache.flink.runtime.operators.DataSourceTask.invoke(DataSourceTask.java:170)
	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.FileNotFoundException: /opt/module/datas/word.txt (没有那个文件或目录)
	at java.io.FileInputStream.open0(Native Method)
	at java.io.FileInputStream.open(FileInputStream.java:195)
	at java.io.FileInputStream.<init>(FileInputStream.java:138)
	at org.apache.flink.core.fs.local.LocalDataInputStream.<init>(LocalDataInputStream.java:50)
	at org.apache.flink.core.fs.local.LocalFileSystem.open(LocalFileSystem.java:142)
	at org.apache.flink.api.common.io.FileInputFormat$InputSplitOpenThread.run(FileInputFormat.java:996)

Two, the cause of the problem

1. The file does not exist in the input path

2. The file of the input path is not on the TaskManager of the flink cluster, but on the JobManager node.

3. 输入路径的文件与Flink集群安装位置在同一级路径下 会有冲突( This is most likely the reason )
insert image description here

3. Solution

1. For a file that does not exist, modify it to the path of the file

2. If the flink input file is a local file, you need to put the file on the TaskManager node (that is, the slaves node)

3. Change the path of the input file
insert image description here

4. Results demonstration

Execute the jar command again:
insert image description here
At this time, it is found that the problem is perfectly solved! ! !

Guess you like

Origin blog.csdn.net/weixin_43520450/article/details/124698750