flink源码阅读---local启动流程

启动脚本:

 start-cluster.sh

bin=`dirname "$0"`
bin=`cd "$bin"; pwd`
. "$bin"/config.sh
# Start the JobManager instance(s)
shopt -s nocasematch
if [[ $HIGH_AVAILABILITY == "zookeeper" ]]; then
    # HA Mode
    readMasters
    echo "Starting HA cluster with ${#MASTERS[@]} masters."
    for ((i=0;i<${#MASTERS[@]};++i)); do
        master=${MASTERS[i]}
        webuiport=${WEBUIPORTS[i]}
        if [ ${MASTERS_ALL_LOCALHOST} = true ] ; then
            "${FLINK_BIN_DIR}"/jobmanager.sh start "${master}" "${webuiport}"
        else
            ssh -n $FLINK_SSH_OPTS $master -- "nohup /bin/bash -l \"${FLINK_BIN_DIR}/jobmanager.sh\" start ${master} ${webuiport} &"
        fi
    done
else
    echo "Starting cluster."
    # Start single JobManager on this machine
    "$FLINK_BIN_DIR"/jobmanager.sh start
fi
shopt -u nocasematch
# Start TaskManager instance(s)
TMSlaves start

可以看到start-cluster.sh调用了jobmanager.sh和taskmanager.sh,其中jobmanager.sh又调用了flink-daemon.sh,flink-daemon.sh接着调用org.apache.flink.runtime.entrypoint.StandaloneSessionClusterEntrypoint,如下图:

case $DAEMON in
    (taskexecutor)
        CLASS_TO_RUN=org.apache.flink.runtime.taskexecutor.TaskManagerRunner
    ;;

    (zookeeper)
        CLASS_TO_RUN=org.apache.flink.runtime.zookeeper.FlinkZooKeeperQuorumPeer
    ;;

    (historyserver)
        CLASS_TO_RUN=org.apache.flink.runtime.webmonitor.history.HistoryServer
    ;;

    (standalonesession)
        CLASS_TO_RUN=org.apache.flink.runtime.entrypoint.StandaloneSessionClusterEntrypoint
    ;;

    (standalonejob)
        CLASS_TO_RUN=org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint
    ;;

    (*)
        echo "Unknown daemon '${DAEMON}'. $USAGE."
        exit 1
    ;;
esac
$JAVA_RUN $JVM_ARGS ${FLINK_ENV_JAVA_OPTS} "${log_setting[@]}" -classpath "`manglePathList "$FLINK_TM_CLASSPATH:$INTERNAL_HADOOP_CLASSPATHS"`" ${CLASS_TO_RUN} "${ARGS[@]}" > "$out" 200<&- 2>&1 < /dev/null &

taskmanager.sh环境变量ENTRYPOINT=taskexecutor,所以调用org.apache.flink.runtime.taskexecutor.TaskManagerRunner;

java程序启动

StandaloneSessionClusterEntrypoint

配置启动如下:

main函数入口

main函数入口,主要是解析传入参数生成configuration配置,然后生成StandaloneSessionClusterEntrypoint对象,调用

StandaloneSessionClusterEntrypoint对象的runClusterEntrypoint方法启动集群,如下图:

启动集群

启动集群调用的clusterEntrypoint.startCluster(),该方法主要是加载plugins、配置文件系统、设置securitycontext、最后调用securitycontext.runSecured()启动集群服务

启动集群服务

集群服务启动调用ClusterEntrypoint对象的runCluster方法,该方法主要初始化服务并启动服务,如下图:

ioExecutor:
 Executors.newFixedThreadPool( ClusterEntrypointUtils.getPoolSize(configuration), new ExecutorThreadFactory("cluster-io"))

haServices = createHaServices(configuration, ioExecutor);

blobServer = new BlobServer(configuration, haServices.createBlobStore());
blobServer.start();
heartbeatServices = createHeartbeatServices(configuration);
metricRegistry = createMetricRegistry(configuration, pluginManager);
processMetricGroup = MetricUtils.instantiateProcessMetricGroup(
   metricRegistry,
   hostname,
   ConfigurationUtils.getSystemResourceMetricsProbingInterval(configuration));
archivedExecutionGraphStore = createSerializableExecutionGraphStore(configuration, commonRpcService.getScheduledExecutor());
DispatcherResourceManagerComponentFactory //启动了{@link Dispatcher}、{@link ResourceManager}和{@link WebMonitorEndpoint}

在同样的过程中

TaskManagerRunner

main函数入口

main函数主要是解析环境变量,获取最大打开文件句柄数,接着调用runTaskManagerSecurely启动taskmanager

启动集群:、

启动taskmanager和jobmanager前两步逻辑一样,最后异步调用runTaskManager启动集群服务

启动集群服务:

至此,flink集群已经启动,可以打开自己的浏览器,输入url地址查看信息如下:

猜你喜欢

转载自blog.csdn.net/wjandy0211/article/details/106525856