Flink-kubernetes-operator

A Kubernetes operator for Apache Flink(https://github.com/apache/flink-kubernetes-operator), it allows users to manage Flink applications and their lifecycle through native k8s tooling like kubectl.

  1. vela addon enable flink-kubernetes-operator
  1. vela addon disable flink-kubernetes-operator

Since this addon dependents fluxcd and cert-manager addon, so will enable them automatically. Check the status of them:

  1. $ vela ls -n vela-system
  2. APP                             COMPONENT               TYPE            TRAITS  PHASE   HEALTHY STATUS                                                          CREATED-TIME                 
  3. addon-cert-manager              cert-manager            helm                    running healthy Fetch repository successfully, Create helm release              2022-06-16 11:50:19 +0800 CST
  4.                                                                                                 successfully                                                                                 
  5. addon-flink-kubernetes-operator flink-namespace         raw                     running healthy                                                                 2022-06-16 11:50:20 +0800 CST
  6. └─                              flink-operator          helm                    running healthy Fetch repository successfully, Create helm release              2022-06-16 11:50:20 +0800 CST
  7.                                                                                                 successfully                                                                                 
  8. addon-fluxcd                    flux-system-namespace   raw                     running healthy                                                                 2022-06-16 11:47:07 +0800 CST
  9. └─                              fluxcd-resources        k8s-objects             running healthy                                                                 2022-06-16 11:47:07 +0800 CST

Show the component type flink-cluster, so we know how to use it in one application. As a Flink user, you can choose the parameter to set for your Flink cluster

  1. vela show flink-cluster
  2. # Specification
  3. # Specification
  4. +--------------+------------------------------------------------------------------------------------------------------+--------+----------+---------+
  5. |     NAME     |                                             DESCRIPTION                                              |  TYPE  | REQUIRED | DEFAULT |
  6. +--------------+------------------------------------------------------------------------------------------------------+--------+----------+---------+
  7. | name         | Specify the flink cluster name.                                                                      | string | true     |         |
  8. | namespace    | Specify the namespace for flink cluster to install.                                                  | string | true     |         |
  9. | nots         | Specify the taskmanager.numberOfTaskSlots, e.g "2".                                                  | string | true     |         |
  10. | flinkVersion | Specify the flink cluster version, e.g "v1_14".                                                      | string | true     |         |
  11. | image        | Specify the image for flink cluster to run, e.g "flink:latest".                                      | string | true     |         |
  12. | jarURI       | Specify the uri for the jar of the flink cluster job, e.g                                            | string | true     |         |
  13. |              | "local:///opt/flink/examples/streaming/StateMachineExample.jar".                                     |        |          |         |
  14. | parallelism  | Specify the parallelism of the flink cluster job, e.g 2.                                             | int    | true     |         |
  15. | upgradeMode  | Specify the upgradeMode of the flink cluster job, e.g "stateless".                                   | string | true     |         |
  16. | replicas     | Specify the replicas of the flink cluster jobManager, e.g 1.                                         | int    | true     |         |
  17. | jmcpu        | Specify the cpu of the flink cluster jobManager, e.g 1.                                              | int    | true     |         |
  18. | jmmem        | Specify the memory of the flink cluster jobManager, e.g "1024m".                                     | string | true     |         |
  19. | tmcpu        | Specify the cpu of the flink cluster taskManager, e.g 1.                                             | int    | true     |         |
  20. | tmmem        | Specify the memory of the flink cluster taskManager, e.g "1024m".                                    | string | true     |         |
  21. +--------------+------------------------------------------------------------------------------------------------------+--------+----------+---------+

First please make sure your cluster already exists namespace flink-cluster.

Then deploy the application:

  1. cat <<EOF | vela up -f -
  2. apiVersion: core.oam.dev/v1beta1
  3. kind: Application
  4. metadata:
  5.   name: flink-app-v1
  6.   namespace: vela-system
  7. spec:
  8.   components:
  9.   - name: my-flink-component
  10.     type: flink-cluster
  11.     properties:
  12.       name: my-flink-cluster
  13.       namespace: flink-cluster
  14.       nots: '2'
  15.       flinkVersion: v1_14
  16.       image: flink:latest
  17.       jarURI: local:///opt/flink/examples/streaming/StateMachineExample.jar
  18.       parallelism: 2
  19.       upgradeMode:  stateless
  20.       replicas: 1
  21.       jmcpu: 1
  22.       jmmem: 1024m
  23.       tmcpu: 1
  24.       tmmem: 1024m

Check the flink cluster

  1. vela ls  -n vela-system | grep app
  2. flink-app-v1                    my-flink-component      flink-cluster                   running healthy                                                               2022-07-30 18:53:34 +0800 CST

Accesss the flink cluster by website using http://localhost:8888

  1. kubectl get svc -n flink-cluster | grep rest
  2. my-flink-cluster-rest   ClusterIP   192.168.149.175   <none>        8081/TCP            17m
  5. kubectl port-forward service/my-flink-cluster-rest 8888:8081 -n flink-cluster
  6. Forwarding from 127.0.0.1:8888 -> 8081

Last updated on Aug 4, 2023 by Daniel Higuero