K8S API for Pipeline

Compared with the Application Workflow, the standalone pipeline has the following characteristics:

  1. It can manage multiple KubeVela Applications across multiple environments.
  2. It is not bound to Applications and can be used independently. For example, it can expand or shrink a set of resources, perform process-oriented canary publishing for an Application, and perform a set of operation and maintenance operations in batches.
  3. It is one-time and does not manage resources. Even if the pipeline is deleted, the created resources will not be deleted.
  4. It uses the same execution engine as the Application Workflow, which completely inherits the features of KubeVela’s lightweight workflow. Compared with the traditional container-based CI pipeline, KubeVela’s pipeline does not depend on containers, No additional computing resources are required.

K8S API for Pipeline - 图1tip

In order to better reuse the existing capabilities and ensure technical consistency, we split the workflow engine part of the original application workflow. Both in-application workflow and pipeline use this workflow engine as the underlying technology implementation. The application workflow is represented by the Workflow field in the application, and the pipeline is represented by the WorkflowRun resource.

This means that most of the workflow steps are common between the two, such as: suspend, notification, send HTTP request, read configuration, etc.

However, in WorkflowRun, there is only the configuration of steps, and no configuration of components, traits, and policies. Therefore, steps related to components/traits/policy can only be used in in-app workflows, such as: deploying/updating components, traits, etc.

WorkflowRun

WorkflowRun is the K8S API for pipeline. You can choose to execute an external Workflow template in the WorkflowRun or execute the steps in the WorkflowRun spec (if you declare both, the step in the WorkflowRun spec will override the content in the template). A WorkflowRun consists of the following:

  1. apiVersion: core.oam.dev/v1alpha1
  2. kind: WorkflowRun
  3. metadata:
  4. name: <name>
  5. namespace: <namespace>
  6. spec:
  7. mode: <optional execute mode for the workflowRun, default execute mode is StepByStep for steps, DAG for subSteps>
  8. steps: <DAG or StepByStep>
  9. subSteps: <DAG or StepByStep>
  10. context:
  11. <optional custom contest values>
  12. workflowRef: <optional external workflow template to run>
  13. workflowSpec: <optional workflow spec to run>
  14. steps:
  15. - name: <name>
  16. type: <type>
  17. dependsOn:
  18. <optional array of step names, specify the dependency for the step>
  19. meta: <optional meta data for the step>
  20. alias: <optional alias of the step>
  21. properties:
  22. <parameter values>
  23. if: <optional if condition to decide whether this step should be executed>
  24. timeout: <optional timeout for the step>
  25. outputs: <optional outputs value>
  26. - name: <name>
  27. valueFrom: <value source of the output>
  28. inputs: <optional inputs value>
  29. - name: <name>
  30. parameterKey: <optional set the inputs data to the steps'parameter>
  31. subSteps:
  32. <optional sub steps if the type of this step is step-group>

Status

WorkflowRun Status

WorkflowRun has the following status:

WorkflowRun StateDescription
executingWhen a step in a WorkflowRun is executing, its status is executing
suspendingWhen a step in a WorkflowRun is suspended, its status is suspending
terminatedWhen a WorkflowRun is terminated, its status is terminated
failedWhen the WorkflowRun is executed completely and a step fails, its status is failed
succeededWhen the WorkflowRun is executed completely and the status of all steps is successful or skipped, its status is succeeded

WorkflowRun Step Status

WorkflowRun steps have the following status:

Step StatusDescription
runningThis step is being executed
succeededThe step is executed successfully
failedThe step failed
skippedThe step is skipped and not executed
pendingThe step is wait for certain conditions to execute, such as: waiting for the inputs

The Failed Reason of WorkflowRun Step

For steps that fail to execute, the message of the step status will display the failed message, and the reason will display the failed reason, which is divided into the following types:

Step Failed ReasonDescription
ExecuteThe step fails in execution
TerminateThe step is terminated
OutputThe step has an error when outputting the Output
FailedAfterRetriesThe Step fails in execution and the retry limit is reached
TimeoutThe step is timeout
Actionop.#Fail is used in the step’s definition

Execution Mode

You can define execution mode in WorkflowRun or Workflow templates:

  1. mode:
  2. steps: <DAG or StepByStep>
  3. subSteps: <DAG or StepByStep>

If not explicitly specified, the WorkflowRun will execute the steps sequentially (StepByStep) and execute sub-steps in parallel (DAG) by default.

K8S API for Pipeline - 图2caution

If you specify the execution mode in both WorkflowRun and Workflow, the mode in WorkflowRun will override the mode in the Workflow template.

Built-in Steps

You can use KubeVela built-in steps that without label: custom.definition.oam.dev/scope: Application in WorkflowRun.

Custom Steps

You can refer to the custom steps documentation to customize your steps.

K8S API for Pipeline - 图3caution

You cannot use application operations.

Core Features

Operate WorkflowRun

K8S API for Pipeline - 图4tip

The vela workflow command can operate both Application Workflow and WorkflowRun. By default, it will look for the application with the same name first, and if it is not found, it will look for WorkflowRun. You can also use --type=workflow to indicate that the operation object is WorkflowRun.

Suspend

If you have an executing WorkflowRun, you can use vela workflow suspend to suspend the workflow.

  1. vela workflow suspend <name>

K8S API for Pipeline - 图5tip

If the workflow has executed completely, using the vela workflow suspend command has no effect.

Resume

When the WorkflowRun is suspended, you can use vela workflow resume command to manually resume the workflow.

  1. vela workflow resume <name>

Terminate

If you have an executing WorkflowRun, you can use vela workflow terminate to terminate the workflow.

  1. vela workflow terminate <name>

Check the Logs

If you want to view the WorkflowRun logs, you can use vela workflow logs command to view the logs.

K8S API for Pipeline - 图6tip

Only steps configured with op.#Log in its definition will have log output.

  1. vela workflow logs <name>

Suspend and Resume

Suspend Manually

Please refer to Operate WorkflowRun.

Suspend Automatically(using suspend step)

  1. apiVersion: core.oam.dev/v1alpha1
  2. kind: WorkflowRun
  3. metadata:
  4. name: suspend
  5. namespace: default
  6. spec:
  7. workflowSpec:
  8. steps:
  9. - name: step1
  10. type: apply-deployment
  11. properties:
  12. image: nginx
  13. - name: step2-suspend
  14. type: suspend
  15. - name: step2
  16. type: apply-deployment
  17. properties:
  18. image: nginx

The WorkflowRun will automatically suspend when the first step is completed, and the third step will not be executed until you continue the WorkflowRun.

Resume Manually

Please refer to Operate WorkflowRun.

Resume Automatically

Configure duration: <duration> in the suspend type of step, when the duration time expires, WorkflowRun will automatically continue to execute.

  1. apiVersion: core.oam.dev/v1alpha1
  2. kind: WorkflowRun
  3. metadata:
  4. name: suspend
  5. namespace: default
  6. spec:
  7. workflowSpec:
  8. steps:
  9. - name: step1
  10. type: apply-deployment
  11. properties:
  12. image: nginx
  13. - name: step2-suspend
  14. type: suspend
  15. properties:
  16. duration: 10s
  17. - name: step2
  18. type: apply-deployment
  19. properties:
  20. image: nginx

When the first step is completed, the WorkflowRun will suspend, and after ten seconds, the WorkflowRun will automatically continue to execute the third step.

Sub Steps

There is a special step type called step-group. When using a step-group type of step, you can declare sub steps in it.

  1. apiVersion: core.oam.dev/v1alpha1
  2. kind: WorkflowRun
  3. metadata:
  4. name: group
  5. namespace: default
  6. spec:
  7. workflowSpec:
  8. steps:
  9. - name: my-group
  10. type: step-group
  11. subSteps:
  12. - name: sub1
  13. type: apply-deployment
  14. properties:
  15. image: nginx
  16. - name: sub2
  17. type: apply-deployment
  18. properties:
  19. image: nginx

Dependency

You can specify dependencies between steps with dependsOn.

  1. apiVersion: core.oam.dev/v1alpha1
  2. kind: WorkflowRun
  3. metadata:
  4. name: dependency
  5. namespace: default
  6. spec:
  7. mode:
  8. steps: DAG
  9. workflowSpec:
  10. steps:
  11. - name: step1
  12. type: apply-deployment
  13. dependsOn:
  14. - step2
  15. - step3
  16. properties:
  17. image: nginx
  18. - name: step2
  19. type: apply-deployment
  20. properties:
  21. image: nginx
  22. - name: step3
  23. type: apply-deployment
  24. properties:
  25. image: nginx

step1 will be executed after step2 and step3 are completed.

Data Passing

Data passing between steps can be done through inputs and outputs. For details, please refer to Input and output between steps.

  1. apiVersion: core.oam.dev/v1alpha1
  2. kind: WorkflowRun
  3. metadata:
  4. name: request-http
  5. namespace: default
  6. spec:
  7. workflowSpec:
  8. steps:
  9. - name: request
  10. type: request
  11. properties:
  12. url: https://api.github.com/repos/kubevela/workflow
  13. outputs:
  14. - name: stars
  15. valueFrom: |
  16. import "strconv"
  17. "Current star count: " + strconv.FormatInt(response["stargazers_count"], 10)
  18. - name: notification
  19. type: notification
  20. inputs:
  21. - from: stars
  22. parameterKey: slack.message.text
  23. properties:
  24. slack:
  25. url:
  26. value: <your slack url>

In this WorkflowRun, the first step will request the GitHub API to get the number of stars in the workflow repository as Output, and then use this Output as Input in the next step to send the star number as the message to Slack.

Timeout

You can specify timeout for a step to indicate the timeout for that step.

timeout follows the duration format, e.g. 30s, 1m, etc. You can refer to Golang’s parseDuration.

  1. apiVersion: core.oam.dev/v1alpha1
  2. kind: WorkflowRun
  3. metadata:
  4. name: timeout
  5. namespace: default
  6. spec:
  7. workflowSpec:
  8. steps:
  9. - name: suspend
  10. type: suspend
  11. timeout: 3s

If the above WorkflowRun is not resumed within three seconds, the suspend step will fail with timeout.

If Conditions

You can use if in a step to determine whether to execute the step.

No If specified

If the step does not specify if, if the step before the step fails to execute, then the step will be skipped and will not be executed.

if: always

With if: always specified in a step, the step will be executed no matter what.

Custom If

You can also write your own judgment logic to determine whether the step should be executed. Note: The value in if will be executed as CUE code. WorkflowRun provides some built-in variables in if, these are:

  • statusstatus contains status information for all workflow steps. You can use status.<step-name>.phase == "succeeded" to determine the status of a step, or you can use the simplified status.<step-name>.succeeded to determine.
  • inputsinputs contains all the inputs parameters of the step. You can use inputs.<input-name> == "value" to get input for the step.
  • context: context contains all the context data of WorkflowRun. You can use context.<context-name> == "value" to get the context of the WorkflowRun.

K8S API for Pipeline - 图7tip

Note that if your step name or inputs name is not a valid CUE variable name (eg: contains -, or starts with a number, etc.), you can refer to it as follows: status["invalid-name"].failed.

  1. apiVersion: core.oam.dev/v1alpha1
  2. kind: WorkflowRun
  3. metadata:
  4. name: if-condition
  5. namespace: default
  6. spec:
  7. workflowSpec:
  8. steps:
  9. - name: suspend
  10. type: suspend
  11. timeout: 3s
  12. - name: my-step
  13. type: apply-deployment
  14. if: status.suspend.failed
  15. properties:
  16. image: nginx
  17. - name: my-step2
  18. type: apply-deployment
  19. if: status.suspend.succecceed
  20. properties:
  21. image: busybox

In the above WorkflowRun, if the suspend step fails due to a timeout, then the my-step step will be executed, otherwise the my-step2 step will be executed.

Custom Context Data

Steps in WorkflowRun have some built-in context data, and you can also declare your custom context parameters in context.

K8S API for Pipeline - 图8tip

If your custom context data has the same name as a built-in context data, the built-in context parameter will be overridden by the custom parameter.

You can control the execution of WorkflowRun in different situations through the combination of conditional if and custom data.

  1. apiVersion: core.oam.dev/v1alpha1
  2. kind: WorkflowRun
  3. metadata:
  4. name: deploy-run
  5. namespace: default
  6. spec:
  7. context:
  8. env: test
  9. workflowRef: deploy-template
  10. ---
  11. apiVersion: core.oam.dev/v1alpha1
  12. kind: Workflow
  13. metadata:
  14. name: deploy-template
  15. namespace: default
  16. steps:
  17. - name: apply
  18. type: apply-deployment
  19. if: context.env == "dev"
  20. properties:
  21. image: nginx
  22. - name: apply-test
  23. type: apply-deployment
  24. if: context.env == "test"
  25. properties:
  26. image: crccheck/hello-world

The above WorkflowRun will refer to the deploy-template Workflow as the execution template. If the env in the context is dev, then the apply step will be executed, otherwise the apply-test step will be executed.

Built-in Context Data

The built-in context data in WorkflowRun are as follows:

Context VariableDescriptionType
context.nameThe name of the WorkflowRunstring
context.namespaceThe namespace of the WorkflowRunstring
context.stepNameThe name of the current stepstring
context.stepSessionIDThe ID of the current stepstring
context.spanIDThe trace ID of current step in this reconcilestring