Pipelines

This document defines Pipelines and their capabilities.

Syntax

To define a configuration file for a Pipeline resource, you can specify the
following fields:

  • Required:
    • [apiVersion][kubernetes-overview] - Specifies the API version, for example
      tekton.dev/v1alpha1.
    • [kind][kubernetes-overview] - Specify the Pipeline resource object.
    • [metadata][kubernetes-overview] - Specifies data to uniquely identify the
      Pipeline resource object, for example a name.
    • [spec][kubernetes-overview] - Specifies the configuration information for
      your Pipeline resource object. In order for a Pipeline to do anything,
      the spec must include:
      • tasks - Specifies which Tasks to run and how to run
        them
  • Optional:

[kubernetes-overview]:
https://kubernetes.io/docs/concepts/overview/working-with-objects/kubernetes-objects/#required-fields

Declared resources

In order for a Pipeline to interact with the outside world, it will probably
need PipelineResources which will be given to
Tasks as inputs and outputs.

Your Pipeline must declare the PipelineResources it needs in a resources
section in the spec, giving each a name which will be used to refer to these
PipelineResources in the Tasks.

For example:

  1. spec:
  2.   resources:
  3.     - name: my-repo
  4.       type: git
  5.     - name: my-image
  6.       type: image

Parameters

Pipelines can declare input parameters that must be supplied to the Pipeline
during a PipelineRun. Pipeline parameters can be used to replace template
values in PipelineTask parameters’ values.

Parameters name are limited to alpha-numeric characters, - and _ and can
only start with alpha characters and _. For example, fooIs-Bar_ is a valid
parameter name, barIsBa$ or 0banana are not.

Usage

The following example shows how Pipelines can be parameterized, and these
parameters can be passed to the Pipeline from a PipelineRun.

Input parameters in the form of ${params.foo} are replaced inside of the
PipelineTask parameters’ values (see also
templating).

The following Pipeline declares an input parameter called ‘context’, and uses
it in the PipelineTask‘s parameter. The description and default fields for
a parameter are optional, and if the default field is specified and this
Pipeline is used by a PipelineRun without specifying a value for ‘context’,
the default value will be used.

  1. apiVersion: tekton.dev/v1alpha1
  2. kind: Pipeline
  3. metadata:
  4.   name: pipeline-with-parameters
  5. spec:
  6.   params:
  7.     - name: context
  8.       description: Path to context
  9.       default: /some/where/or/other
  10.   tasks:
  11.     - name: build-skaffold-web
  12.       taskRef:
  13.         name: build-push
  14.       params:
  15.         - name: pathToDockerFile
  16.           value: Dockerfile
  17.         - name: pathToContext
  18.           value: "${params.context}"

The following PipelineRun supplies a value for context:

  1. apiVersion: tekton.dev/v1alpha1
  2. kind: PipelineRun
  3. metadata:
  4.   name: pipelinerun-with-parameters
  5. spec:
  6.   pipelineRef:
  7.     name: pipeline-with-parameters
  8.   params:
  9.     - name: "context"
  10.       value: "/workspace/examples/microservices/leeroy-web"

Pipeline Tasks

A Pipeline will execute a graph of Tasks (see
ordering for how to express this graph). At a minimum, this
declaration must include a reference to the Task:

  1. tasks:
  2.   - name: build-the-image
  3.     taskRef:
  4.       name: build-push

Declared PipelineResources can be given to Tasks in
the Pipeline as inputs and outputs, for example:

  1. spec:
  2.   tasks:
  3.     - name: build-the-image
  4.       taskRef:
  5.         name: build-push
  6.       resources:
  7.         inputs:
  8.           - name: workspace
  9.             resource: my-repo
  10.         outputs:
  11.           - name: image
  12.             resource: my-image

Parameters can also be provided:

  1. spec:
  2.   tasks:
  3.     - name: build-skaffold-web
  4.       taskRef:
  5.         name: build-push
  6.       params:
  7.         - name: pathToDockerFile
  8.           value: Dockerfile
  9.         - name: pathToContext
  10.           value: /workspace/examples/microservices/leeroy-web

from

Sometimes you will have Pipeline Tasks that need to take as
input the output of a previous Task, for example, an image built by a previous
Task.

Express this dependency by adding from on PipelineResources
that your Tasks need.

  • The (optional) from key on an input source defines a set of previous
    PipelineTasks (i.e. the named instance of a Task) in the Pipeline
  • When the from key is specified on an input source, the version of the
    resource that is from the defined list of tasks is used
  • from can support fan in and fan out
  • The from clause expresses ordering, i.e. the
    Pipeline Task which provides the PipelineResource must run
    before the Pipeline Task which needs that PipelineResource as an input
    • The name of the PipelineResource must correspond to a PipelineResource
      from the Task that the referenced PipelineTask gives as an output

For example see this Pipeline spec:

  1. - name: build-app
  2.   taskRef:
  3.     name: build-push
  4.   resources:
  5.     outputs:
  6.       - name: image
  7.         resource: my-image
  8. - name: deploy-app
  9.   taskRef:
  10.     name: deploy-kubectl
  11.   resources:
  12.     inputs:
  13.       - name: my-image
  14.         from:
  15.           - build-app

The resource my-image is expected to be given to the deploy-app Task from
the build-app Task. This means that the PipelineResource my-image must
also be declared as an output of build-app.

This also means that the build-app Pipeline Task will run before deploy-app,
regardless of the order they appear in the spec.

runAfter

Sometimes you will need to have Pipeline Tasks that need to
run in a certain order, but they do not have an explicit
output to input dependency (which is
expressed via from). In this case you can use runAfter to indicate
that a Pipeline Task should be run after one or more previous Pipeline Tasks.

For example see this Pipeline spec:

  1. - name: test-app
  2.   taskRef:
  3.     name: make-test
  4.   resources:
  5.     inputs:
  6.       - name: my-repo
  7. - name: build-app
  8.   taskRef:
  9.     name: kaniko-build
  10.   runAfter:
  11.     - test-app
  12.   resources:
  13.     inputs:
  14.       - name: my-repo

In this Pipeline, we want to test the code before we build from it, but there
is no output from test-app, so build-app uses runAfter to indicate that
test-app should run before it, regardless of the order they appear in the
spec.

Ordering

The Pipeline Tasks in a Pipeline can be connected and run
in a graph, specifically a Directed Acyclic Graph or DAG. Each of the Pipeline
Tasks is a node, which can be connected (i.e. a Graph) such that one will run
before another (i.e. Directed), and the execution will eventually complete
(i.e. Acyclic, it will not get caught in infinite loops).

This is done using:

For example see this Pipeline spec:

  1. - name: lint-repo
  2.   taskRef:
  3.     name: pylint
  4.   resources:
  5.     inputs:
  6.       - name: my-repo
  7. - name: test-app
  8.   taskRef:
  9.     name: make-test
  10.   resources:
  11.     inputs:
  12.       - name: my-repo
  13. - name: build-app
  14.   taskRef:
  15.     name: kaniko-build-app
  16.   runAfter:
  17.     - test-app
  18.   resources:
  19.     inputs:
  20.       - name: my-repo
  21.     outputs:
  22.       - name: image
  23.         resource: my-app-image
  24. - name: build-frontend
  25.   taskRef:
  26.     name: kaniko-build-frontend
  27.   runAfter:
  28.     - test-app
  29.   resources:
  30.     inputs:
  31.       - name: my-repo
  32.     outputs:
  33.       - name: image
  34.         resource: my-frontend-image
  35. - name: deploy-all
  36.   taskRef:
  37.     name: deploy-kubectl
  38.   resources:
  39.     inputs:
  40.       - name: my-app-image
  41.         from:
  42.           - build-app
  43.       - name: my-frontend-image
  44.         from:
  45.           - build-frontend

This will result in the following execution graph:

  1.         |            |
  2.         v            v
  3.      test-app    lint-repo
  4.     /        \
  5.    v          v
  6. build-app  build-frontend
  7.    \          /
  8.     v        v
  9.     deploy-all
  1. The lint-repo and test-app Pipeline Tasks will begin executing
    simultaneously. (They have no from or runAfter clauses.)
  2. Once test-app completes, both build-app and build-frontend will begin
    executing simultaneously (both runAfter test-app).
  3. When both build-app and build-frontend have completed, deploy-all will
    execute (it requires PipelineResources from both Pipeline Tasks).
  4. The entire Pipeline will be finished executing after lint-repo and
    deploy-all have completed.

Examples

For complete examples, see
the examples folder.

Except as otherwise noted, the content of this page is licensed under the
Creative Commons Attribution 4.0 License,
and code samples are licensed under the
Apache 2.0 License.