Flink-kubernetes-operator

A Kubernetes operator for Apache Flink(https://github.com/apache/flink-kubernetes-operator), it allows users to manage Flink applications and their lifecycle through native k8s tooling like kubectl.

Install

  1. vela addon enable flink-kubernetes-operator

Uninstall

  1. vela addon disable flink-kubernetes-operator

Since this addon dependents fluxcd and cert-manager addon, so will enable them automatically. Check the status of them:

  1. $ vela ls -n vela-system
  2. APP                             COMPONENT               TYPE            TRAITS  PHASE   HEALTHY STATUS                                                          CREATED-TIME                 
  3. addon-cert-manager              cert-manager            helm                    running healthy Fetch repository successfully, Create helm release              2022-06-16 11:50:19 +0800 CST
  4.                                                                                                 successfully                                                                                 
  5. addon-flink-kubernetes-operator flink-namespace         raw                     running healthy                                                                 2022-06-16 11:50:20 +0800 CST
  6. └─                              flink-operator          helm                    running healthy Fetch repository successfully, Create helm release              2022-06-16 11:50:20 +0800 CST
  7.                                                                                                 successfully                                                                                 
  8. addon-fluxcd                    flux-system-namespace   raw                     running healthy                                                                 2022-06-16 11:47:07 +0800 CST
  9. └─                              fluxcd-resources        k8s-objects             running healthy                                                                 2022-06-16 11:47:07 +0800 CST

Show the component type flink-cluster, so we know how to use it in one application. As a Flink user, you can choose the parameter to set for your Flink cluster

  1. vela show flink-cluster
  2. # Properties
  3. +--------------+-------------+--------+----------+---------------------------------------------------------------+
  4. |     NAME     | DESCRIPTION |  TYPE  | REQUIRED |                            DEFAULT                            |
  5. +--------------+-------------+--------+----------+---------------------------------------------------------------+
  6. | name         |             | string | true     |                                                               |
  7. | namespace    |             | string | true     |                                                               |
  8. | nots         |             | string | true     |                                                             2 |
  9. | flinkVersion |             | string | true     | v1_14                                                         |
  10. | image        |             | string | true     | flink:latest                                                  |
  11. | jarURI       |             | string | true     | local:///opt/flink/examples/streaming/StateMachineExample.jar |
  12. | parallelism  |             | int    | true     |                                                             2 |
  13. | upgradeMode  |             | string | true     | stateless                                                     |
  14. | replicas     |             | int    | true     |                                                             1 |
  15. | jmcpu        |             | int    | true     |                                                             1 |
  16. | jmmem        |             | string | true     | 1024m                                                         |
  17. | tmcpu        |             | int    | true     |                                                             1 |
  18. | tmmem        |             | string | true     | 1024m                                                         |
  19. +--------------+-------------+--------+----------+---------------------------------------------------------------+

First please make sure your cluster already exists namespace flink-home.

Then deploy the application:

  1. cat <<EOF | vela up -f -
  2. apiVersion: core.oam.dev/v1beta1
  3. kind: Application
  4. metadata:
  5.    name: flink-app-v1
  6. spec:
  7. components:
  8.   - name: my-flink-component
  9.     type: flink-cluster
  10.     properties:
  11.       name: my-flink-cluster
  12.       namespace: flink-home
  13. EOF

Check all the related resources:

  1. vela status flink-app-v1 
  2. About:
  4. Name:         flink-app-v1
  5. Created at:   2022-04-22 17:33:51 +0800 CST
  6. Status:       running
  8. Workflow:
  10. mode: DAG
  11. finished: true
  12. Suspend: false
  13. Terminated: false
  14. Steps
  15. - id:n6na24x6dr
  16.   name:my-flink-component
  17.   type:apply-component
  18.   phase:succeeded
  19.   message:
  21. Services:
  23. - Name: my-flink-component
  24.   Cluster: local  Namespace: default
  25.   Type: flink-cluster
  26.   Healthy
  27.   No trait applied

You can see you first flink-cluster application is running!