Run Flink tasks on IDEA

IDEA is a commonly used IDE. If the flink task code we write can be run directly on IDEA, it will bring great convenience to learning and development. For example, after the code is changed, it will run immediately without deployment, breakpoints, and single-step debugging

Environmental information

Computer: 2019 version 13-inch MacBook Pro, 2.3 GHz quad-core Intel Core i5, 8 GB 2133 MHz LPDDR3
Operating system: macOS Catalina 10.15.3
JDK：1.8.0_211
Maven：3.6.0
IDEA：2018.3.5 (Ultimate Edition)
Flink version: 1.9.2

About genuine IDEA

For how to use genuine IDEA for free, please refer to "Free Application and Use of IntelliJ IDEA Commercial License Guide"

Development

First make sure that maven is installed on your computer;
Create a maven project for the flink task, find a clean directory and execute the following commands:

mvn \
archetype:generate \
-DarchetypeGroupId=org.apache.flink \
-DarchetypeArtifactId=flink-quickstart-java \
-DarchetypeVersion=1.9.2

Follow the prompts to enter groupId, artifactId, version, package, etc .:
Now that the maven project has been generated, use IDEA to import this project, as shown below:
Import as maven:
What it looks like after successful import:
Modify StreamingJob.java to add the function of monitoring the local port 18081 , get the character string to do word count operation, and download the complete code here: https://raw.githubusercontent.com/zq2599/blog_demos/master/files/StreamingJob.java
The current code can be run, but the flink webpage is not yet accessible, the following error message will be displayed, and you need to continue to make some settings:
Download the flink-1.9.2 installation package, there is a jar used by the flink web service, the address is: https://www-eu.apache.org/dist/flink/flink-1.9.2/flink-1.9. 2-bin-scala_2.11.tgz
flink-1.9.2-bin-scala_2.11.tgz download and unzip, there is a flink-dist_2.11-1.9.2.jar file in the lib directory , remember the location of this file, you will use it later;
Go back to IDEA, right-click on the project, and click the menu Open Module Settings :
Do the following in the pop-up window, the purpose is to add jar to the project:
A window for selecting a file pops up, please select the file flink-dist_2.11-1.9.2.jar that you just prepared :
The setup work has been completed. Since the job of StreamingJob is to read the data on port 18081 of the machine, we need to start the service on port 18081, otherwise the StreamingJob will not be connected to the port when it is running. Open a console and execute the command: nc -l 18081
Now you can run the StreamingJob, as shown in the figure below, right-click on the StreamingJob, select Run 'StreamingJob.main ()': you can start the flink task, if you want to interrupt the debugging, please select Debug 'StreamingJob.main ()'
Go back to the console where you just executed the nc command, enter some strings, such as "aaa bbb aaa", and then press Enter;
Going back to IDEA, we can see that StreamingJob has read the data from port 18081 and output the statistical results:
Search for the keyword localhost in the IDEA console , as shown in the red box below, you can find the port of the flink webpage, I am here 62641 :
The browser visits http: // localhost: 62641 , as shown in the figure below, it can be seen that the flink webpage has been displayed normally, and the running tasks can also be seen:

At this point, the simplest IDEA's actual combat to run the flink task is completed, if you are also learning flink, hope this article can give you some reference;