2. In which environments can flink computing tasks run

Table of contents

1. Description

2. Initialize the execution environment object

2.1 getExecutionEnvironment (recommended)

2.2 createLocalEnvironment (local environment object)

2.3 createRemoteEnvironment (remote environment object)

---

1. Description

Flink computing tasks are codes written in specific languages ​​(java, scala, python, sql) according to business requirements

Finally, the written code needs to be sent to the runnable environment to calculate the required results

Flink computing tasks support running in multiple environments:

        Cluster environment: Yarn, Standalone, K8s

        local environment (debugging use)

This requires finding a `suitable` execution environment object when submitting Flink computing tasks

---

2. Initialize the execution environment object

2.1 getExecutionEnvironment (recommended)

grammar:

Automatically obtain the appropriate execution environment object through the configuration file object.
           If it is executed in the IDE or as a general java program, it will return to the local execution environment.
           If you submit the jar package to the cluster, it will return to the cluster execution environment.

public static StreamExecutionEnvironment getExecutionEnvironment() {
    return getExecutionEnvironment(new Configuration());
}

---

2.2 createLocalEnvironment (local environment object)

grammar:

Tips for creating a local execution environment object
:
       1. Execute in a multi-threaded manner in the local JVM
       2. When the degree of parallelism is not specified, use the number of CPU cores of the current machine as the degree of parallelism

---

2.3 createRemoteEnvironment (remote environment object)

grammar:

Tips for creating remote execution environment objects :
    1. When the degree of parallelism is not specified, the degree of parallelism set in flink-conf.yaml will be used parallelism.default=1

---

3. Code example

Development language: Java1.8

Flink version: 1.17.0

package com.baidu.datastream.env;

import org.apache.flink.streaming.api.environment.LocalStreamEnvironment;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;

/*
 * TODO 思考：flink的计算任务(flink application) 可以在那些环境运行呢？
 *      1.集群环境：yarn、Standalone、K8s
 *      2.本地环境：本地的JVM中
 * */

public class ExecutionEnvironment {
    public static void main(String[] args) {
        // 方式1
        //getExecutionEnvironment();
        // 方式2
        //createLocalEnvironment();
        // 方式3
        createRemoteEnvironment();
    }

    // 方式1(推荐): getExecutionEnvironment 方法
    public static void getExecutionEnvironment() {
        /*
         * TODO 根据运行的上下文(配置文件)，来判断当前运行环境
         *     如果在IDE中执行 或者 作为一般的java程序执行 ，则返回 本地执行环境
         *     如果你将 jar包提交到集群， 则返回 集群执行环境
         *
         * */
        StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
        System.out.println("并行度: " + env.getParallelism());
    }

    // 方式2(不推荐): createLocalEnvironment 方法
    public static void createLocalEnvironment() {
        /*
         * TODO 返回本地执行环境
         * */
        LocalStreamEnvironment localEnv = StreamExecutionEnvironment.createLocalEnvironment();
        System.out.println("并行度: " + localEnv.getParallelism());
    }

    // 方式3(不推荐): createRemoteEnvironment 方法
    public static void createRemoteEnvironment() {
        /*
         * TODO 返回集群执行环境
         * */
        StreamExecutionEnvironment remoteEnv = StreamExecutionEnvironment.createRemoteEnvironment(
                "localhost"
                , 8081
                , 8
                , "../FlinkAPI/target/FlinkAPI-1.0-SNAPSHOT.jar"
        );
        System.out.println("并行度: " + remoteEnv.getParallelism());
    }
}