Based k8s1.16 build flink cluster (two)

Disclaimer: This article is a blogger original article, follow the CC 4.0 BY-SA copyright agreement, reproduced, please attach the original source link and this statement.

This link: https://blog.csdn.net/RivenDong/article/details/102712531

1. Experimental environment

Kubernetes clusters:
three 4G virtual machine, CentOS 7.7 release
version kubernetes cluster:
Kubernetes v1.16.0

k8s 1.16 and 1.15 k8s difference:
after k8s 1.16 version of Deployment api change, so do it again

2. Cluster Setup

2.1 k8s1.16 version of Yaml configuration

After k8s 1.16 version, Deployment of apiVersion becomes the apps / v1, and the need to join in the spec selector.matchLabels.app

2.1.1 JobManager Yaml configuration

The main components of a mirror provided JobManager run parameters, including parameters JobManager itself, e.g. RPC port configuration information:

apiVersion: apps/v1
kind: Deployment
metadata:
    name: flink-jobmanager
spec:
    selector:
        matchLabels:
            app: flink
    replicas: 1
    template:
        metadata:
            labels:
                app: flink
                component: jobmanager
        spec:
            containers:
            - name: jobmanager
              image: flink:1.9
              args:
              - jobmanager
              ports:
              - containerPort: 6123
                name: rpc
              - containerPort: 6124
                name: blob
              - containerPort: 6125
                name: query
              - containerPort: 8081
                name: ui
              env:
              - name: JOB_MANAGER_RPC_ADDRESS
                value: flink-jobmanager

2.1.2 TaskManager Yaml configuration

The main component providing TaskManager running parameters, as well as its own parameters TaskManager, e.g. RPC ports configuration information:

apiVersion: apps/v1
kind: Deployment
metadata:
    name: flink-taskmanager
spec:
    selector:
        matchLabels:
            app: flink
    replicas: 3
    template:
        metadata:
            labels:
                app: flink
                component: taskmanager
        spec:
            containers:
            - name: taskmanager
              image: flink:1.9
              args:
              - taskmanager
              ports:
              - containerPort: 6121
                name: data
              - containerPort: 6122
                name: rpc
              - containerPort: 6125
                name: query
              env:
              - name: JOB_MANAGER_RPC_ADDRESS
                value: flink-jobmanager

2.1.3 JobManagerServices Configuration

The main provision of external cluster RestApi for Flink Session UI and address, so that users can access the cluster Flink UI and acquire tasks and monitoring information, the configuration file as follows:

apiVersion: v1
kind: Service
metadata:
    name: flink-jobmanager
spec:
    type: NodePort
    ports:
    - name: rpc
      port: 6123 
    - name: blob
      port: 6124
    - name: query
      port: 6125
    - name: ui
      port: 8081
      nodePort: 30001
    selector:
        app: flink
        component: jobmanager

2.2 Starting Flink Session Cluster

When the individual components of the service profile defined, you can Kubectl by using the following command to create Flink Session Cluster, after the completion of the cluster can be configured to start by JobManagerServices the WebUI port access Flink Web page.

kubectl create -f jobmanager-service.yaml

kubectl create -f jobmanager-deployment.yaml

kubectl create -f taskmanager-deployment.yaml

Use the following command to view the pod:

kubectl get pods -o wide

Flink then access the Web UI interface:

Stop Flink Session Cluster 2.3

kubectl delete-f jobmanager-service.yaml

kubectl delete -f jobmanager-deployment.yaml

kubectl delete -f taskmanager-deployment.yaml