PipelineResources

PipelineResources in a pipeline are the set of objects that are going to beused as inputs to a Task and can be output by a Task.

A Task can have multiple inputs and outputs.

For example:

• A Task’s input could be a GitHub source which contains your applicationcode.
• A Task’s output can be your application container image which can be thendeployed in a cluster.
• A Task’s output can be a jar file to be uploaded to a storage bucket.

Syntax

To define a configuration file for a PipelineResource, you can specify thefollowing fields:

• Required:
  • apiVersion - Specifies the API version, forexample tekton.dev/v1alpha1.
  • kind - Specify the PipelineResource resourceobject.
  • metadata - Specifies data to uniquely identifythe PipelineResource object, for example a name.
  • spec - Specifies the configuration informationfor your PipelineResource resource object.
    • type - Specifies the type of thePipelineResource
• Optional:
  • description - Description of the Resource.
  • params - Parameters which are specific to each typeof PipelineResource
  • optional - Boolean flag to mark a resourceoptional (by default, optional is set to false making resourcesmandatory).

Using Resources

Resources can be used in Tasks andConditions.

Input resources, like source code (git) or artifacts, are dumped at path/workspace/task_resource_name within a mountedvolume and are availableto all steps of your Task. The path that the resources are mountedat can beoverridden with the targetPath field.Steps can use the pathvariable substitution key torefer to the local path to the mounted resource.

Variable substitution

Task and Condition specs can refer resource params as well as predefinedvariables such as path using the variable substitution syntax below where<name> is the resource’s name and <key> is one of the resource’s params:

In Task Spec:

For an input resource in a Task spec: shell $(resources.inputs.<name>.<key>)

Or for an output resource:

$(outputs.resources.<name>.<key>)

In Condition Spec:

Input resources can be accessed by:

$(resources.<name>.<key>)

Accessing local path to resource

The path key is predefined and refers to the local path to a resource on themounted volume shell $(resources.inputs.<name>.path)

Controlling where resources are mounted

The optional field targetPath can be used to initialize a resource in aspecific directory. If targetPath is set, the resource will be initializedunder /workspace/targetPath. If targetPath is not specified, the resourcewill be initialized under /workspace. The following example demonstrates howgit input repository could be initialized in $GOPATH to run tests:

apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  name: task-with-input
  namespace: default
spec:
  resources:
    inputs:
      - name: workspace
        type: git
        targetPath: go/src/github.com/tektoncd/pipeline
  steps:
    - name: unit-tests
      image: golang
      command: ["go"]
      args:
        - "test"
        - "./..."
      workingDir: "/workspace/go/src/github.com/tektoncd/pipeline"
      env:
        - name: GOPATH
          value: /workspace/go

Overriding where resources are copied from

When specifying input and output PipelineResources, you can optionally specifypaths for each resource. paths will be used by TaskRun as the resource’snew source paths i.e., copy the resource from a specified list of paths.TaskRun expects the folder and contents to be already present in specifiedpaths. The paths feature could be used to provide extra files or alteredversion of existing resources before the execution of steps.

The output resource includes the name and reference to the pipeline resource andoptionally paths. paths will be used by TaskRun as the resource’s newdestination paths i.e., copy the resource entirely to specified paths. TaskRunwill be responsible for the creation of required directories and contenttransition. The paths feature could be used to inspect the results ofTaskRun after the execution of steps.

paths feature for input and output resources is heavily used to pass the sameversion of resources across tasks in context of PipelineRun.

In the following example, Task and TaskRun are defined with an inputresource, output resource and step, which builds a war artifact. After theexecution of TaskRun(volume-taskrun), custom volume will have the entireresource java-git-resource (including the war artifact) copied to thedestination path /custom/workspace/.

apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  name: volume-task
  namespace: default
spec:
  resources:
    inputs:
      - name: workspace
        type: git
    outputs:
      - name: workspace
  steps:
    - name: build-war
      image: objectuser/run-java-jar #https://hub.docker.com/r/objectuser/run-java-jar/
      command: jar
      args: ["-cvf", "projectname.war", "*"]
      volumeMounts:
        - name: custom-volume
          mountPath: /custom

apiVersion: tekton.dev/v1beta1
kind: TaskRun
metadata:
  name: volume-taskrun
  namespace: default
spec:
  taskRef:
    name: volume-task
  resources:
    inputs:
      - name: workspace
        resourceRef:
          name: java-git-resource
    outputs:
      - name: workspace
        paths:
          - /custom/workspace/
        resourceRef:
          name: java-git-resource
  podTemplate:
    volumes:
      - name: custom-volume
        emptyDir: {}

Resource Status

When resources are bound inside a TaskRun, they can include extra informationin the TaskRun Status.ResourcesResult field. This information can be usefulfor auditing the exact resources used by a TaskRun later. Currently the Imageand Git resources use this mechanism.

For an example of what this output looks like:

resourcesResult:
- key: digest
  value: sha256:a08412a4164b85ae521b0c00cf328e3aab30ba94a526821367534b81e51cb1cb
  resourceRef:
    name: skaffold-image-leeroy-web

Description

The description field is an optional field and can be used to provide description of the Resource.

Optional Resources

By default, a resource is declared as mandatory unless optional is set to truefor that resource. Resources declared as optional in a Task does not have bespecified in TaskRun.

apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  name: task-check-optional-resources
spec:
  resources:
    inputs:
      - name: git-repo
        type: git
        optional: true

Similarly, resources declared as optional in a Pipeline does not have to bespecified in PipelineRun.

apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: pipeline-build-image
spec:
  resources:
    - name: workspace
      type: git
      optional: true
  tasks:
    - name: check-workspace
...

You can refer to different examples demonstrating usage of optional resources inTask, Condition, and Pipeline:

• Task
• Cluster Task
• Condition
• Pipeline

Resource Types

Git Resource

The git resource represents a git repository, thatcontains the source code to be built by the pipeline. Adding the git resourceas an input to a Task will clone this repository and allow the Task toperform the required actions on the contents of the repo.

To create a git resource using the PipelineResource CRD:

apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
  name: wizzbang-git
  namespace: default
spec:
  type: git
  params:
    - name: url
      value: https://github.com/wizzbangcorp/wizzbang.git
    - name: revision
      value: master

Params that can be added are the following:

• url: represents the location of the git repository, you can use this tochange the repo, e.g. to use a fork
• revision: Git revision (branch, tag, commit SHA or ref) toclone. You can use this to control what commit or branchis used. If no revision is specified, the resource will default to latestfrom master.
• submodules: defines if the resource should initialize and fetch thesubmodules, value is either true or false. If not specified, this willdefault to true
• depth: performs a shallow clone where only the most recentcommit(s) will be fetched. If set to '0', all commits will be fetched. Ifnot specified, the default depth is 1.
• sslVerify: defines if http.sslVerify should be setto true or false in the global git config. Defaults to true ifomitted. When used as an input, the Git resource includes the exact commit fetched in theresourceResults section of the taskRun’s status object:

resourceResults:
- key: commit
  value: 6ed7aad5e8a36052ee5f6079fc91368e362121f7
  resourceRef:
    name: skaffold-git

Using a fork

The Url parameter can be used to point at any git repository, for example touse a GitHub fork at master:

spec:
  type: git
  params:
    - name: url
      value: https://github.com/bobcatfish/wizzbang.git

Using a branch

The revision can be anygit commit-ish (revision).You can use this to create a git PipelineResource that points at a branch, forexample:

spec:
  type: git
  params:
    - name: url
      value: https://github.com/wizzbangcorp/wizzbang.git
    - name: revision
      value: some_awesome_feature

To point at a pull request, you can usethe pull requests’s branch:

spec:
  type: git
  params:
    - name: url
      value: https://github.com/wizzbangcorp/wizzbang.git
    - name: revision
      value: refs/pull/52525/head

Using HTTP/HTTPS Proxy

The httpProxy and httpsProxy parameter can be used to proxy non-SSL/SSL requests, for example to use an enterpriseproxy server for SSL requests:

spec:
  type: git
  params:
    - name: url
      value: https://github.com/bobcatfish/wizzbang.git
    - name: httpsProxy
      value: "my-enterprise.proxy.com"

Using No Proxy

The noProxy parameter can be used to opt out of proxying, for example, to not proxy HTTP/HTTPS requests tono.proxy.com:

spec:
  type: git
  params:
    - name: url
      value: https://github.com/bobcatfish/wizzbang.git
    - name: noProxy
      value: "no.proxy.com"

Note: httpProxy, httpsProxy, and noProxy are all optional but no validation done if all three are specified.

Pull Request Resource

The pullRequest resource represents a pull request event from a source controlsystem.

Adding the Pull Request resource as an input to a Task will populate theworkspace with a set of files containing generic pull request related metadatasuch as base/head commit, comments, and labels.

The payloads will also contain links to raw service-specific payloads whereappropriate.

Adding the Pull Request resource as an output of a Task will update the sourcecontrol system with any changes made to the pull request resource during thepipeline.

Example file structure:

/workspace/
/workspace/<resource>/
/workspace/<resource>/labels/
/workspace/<resource>/labels/<label>
/workspace/<resource>/status/
/workspace/<resource>/status/<status>
/workspace/<resource>/comments/
/workspace/<resource>/comments/<comment>
/workspace/<resource>/head.json
/workspace/<resource>/base.json
/workspace/<resource>/pr.json

More details:

Labels are empty files, named after the desired label string.

Statuses describe pull request statuses. It is represented as a set of jsonfiles.

References (head and base) describe Git references. They are represented as aset of json files.

Comments describe a pull request comment. They are represented as a set of jsonfiles.

Other pull request information can be found in pr.json. This is a read-onlyresource. Users should use other subresources (labels, comments, etc) tointeract with the PR.

For an example of the output this resource provides, seeexample.

To create a pull request resource using the PipelineResource CRD:

apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
  name: wizzbang-pr
  namespace: default
spec:
  type: pullRequest
  params:
    - name: url
      value: https://github.com/wizzbangcorp/wizzbang/pulls/1
  secrets:
    - fieldName: authToken
      secretName: github-secrets
      secretKey: token
---
apiVersion: v1
kind: Secret
metadata:
  name: github-secrets
type: Opaque
data:
  token: github_personal_access_token_secret # in base64 encoded form

Params that can be added are the following:

• url: represents the location of the pull request to fetch.
• provider: represents the SCM provider to use. This will be “guessed” basedon the url if not set. Valid values are github or gitlab today.
• insecure-skip-tls-verify: represents whether to skip verification of certificatesfrom the git server. Valid values are "true" or "false", the default being"false".

Statuses

The following status codes are available to use for the Pull Request resource:https://godoc.org/github.com/jenkins-x/go-scm/scm#State

Pull Request

The pullRequest resource will look for GitHub or GitLab OAuth authenticationtokens in spec secrets with a field name called authToken.

URLs should be of the form: https://github.com/tektoncd/pipeline/pull/1

Self hosted / Enterprise instances

The PullRequest resource works with self hosted or enterprise GitHub/GitLabinstances. Simply provide the pull request URL and set the provider parameter.If you need to skip certificate validation set the insecure-skip-tls-verifyparameter to "true".

apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
  name: wizzbang-pr
  namespace: default
spec:
  type: pullRequest
  params:
    - name: url
      value: https://github.example.com/wizzbangcorp/wizzbang/pulls/1
    - name: provider
      value: github

Image Resource

An image resource represents an image that lives in a remote repository. It isusually used as a Task output for Tasks that buildimages. This allows the same Tasks to be used to generically push to anyregistry.

Params that can be added are the following:

• url: The complete path to the image, including the registry and the imagetag
• digest: Theimage digestwhich uniquely identifies a particular build of an image with a particulartag. While this can be provided as a parameter, there is not yet a way toupdate this value after an image is built, but this is planned in#216. For example:

apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
  name: kritis-resources-image
  namespace: default
spec:
  type: image
  params:
    - name: url
      value: gcr.io/staging-images/kritis

Surfacing the image digest built in a task

To surface the image digest in the output of the taskRun the builder toolshould produce this information in aOCI Image Layoutindex.json file. This file should be placed on a location as specified in thetask definition under the default resource directory, or the specifiedtargetPath. If there is only one image in the index.json file, the digest ofthat image is exported; otherwise, the digest of the whole image index would beexported. For example this build-push task defines the outputImageDir for thebuiltImage resource in /workspace/buildImage

apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  name: build-push
spec:
  resources:
    inputs:
      - name: workspace
        type: git
    outputs:
      - name: builtImage
        type: image
        targetPath: /workspace/builtImage
  steps: ...

If no value is specified for targetPath, it will default to/workspace/output/{resource-name}.

Please check the builder tool used on how to pass this path to create theoutput file.

The taskRun will include the image digest in the resourcesResult field thatis part of the taskRun.Status

for example:

status:
    ...
    resourcesResult:
    - key: "digest"
      value: "sha256:eed29cd0b6feeb1a92bc3c4f977fd203c63b376a638731c88cacefe3adb1c660"
      resourceRef:
        name: skaffold-image-leeroy-web
    ...

If the index.json file is not produced, the image digest will not be includedin the taskRun output.

Cluster Resource

A cluster resource represents a Kubernetes cluster other than the currentcluster Tekton Pipelines is running on. A common use case for this resource isto deploy your application/function on different clusters.

The resource will use the provided parameters to create akubeconfigfile that can be used by other steps in the pipeline Task to access the targetcluster. The kubeconfig will be placed in/workspace/<your-cluster-name>/kubeconfig on your Task container

The Cluster resource has the following parameters:

• url (required): Host url of the master node
• username (required): the user with access to the cluster
• password: to be used for clusters with basic auth
• namespace: The namespace to target in the cluster
• token: to be used for authentication, if present will be used ahead of thepassword
• insecure: to indicate server should be accessed without verifying the TLScertificate.
• cadata (required): holds PEM-encoded bytes (typically read from a rootcertificates bundle).

Note: Since only one authentication technique is allowed per user, either atoken or a password should be provided, if both are provided, the passwordwill be ignored.

The following example shows the syntax and structure of a cluster resource:

apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
  name: test-cluster
spec:
  type: cluster
  params:
    - name: url
      value: https://10.10.10.10 # url to the cluster master node
    - name: cadata
      value: LS0tLS1CRUdJTiBDRVJ.....
    - name: token
      value: ZXlKaGJHY2lPaU....

For added security, you can add the sensitive information in a KubernetesSecret and populatethe kubeconfig from them.

For example, create a secret like the following example:

apiVersion: v1
kind: Secret
metadata:
  name: target-cluster-secrets
data:
  cadatakey: LS0tLS1CRUdJTiBDRVJUSUZ......tLQo=
  tokenkey: ZXlKaGJHY2lPaUpTVXpJMU5pSXNJbX....M2ZiCg==

and then apply secrets to the cluster resource

apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
  name: test-cluster
spec:
  type: cluster
  params:
    - name: url
      value: https://10.10.10.10
    - name: username
      value: admin
  secrets:
    - fieldName: token
      secretKey: tokenKey
      secretName: target-cluster-secrets
    - fieldName: cadata
      secretKey: cadataKey
      secretName: target-cluster-secrets

Example usage of the cluster resource in a Task, usingvariable substitution:

apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  name: deploy-image
  namespace: default
spec:
  resources:
    inputs:
      - name: workspace
        type: git
      - name: dockerimage
        type: image
      - name: testcluster
        type: cluster
  steps:
    - name: deploy
      image: image-with-kubectl
      command: ["bash"]
      args:
        - "-c"
        - kubectl --kubeconfig
          /workspace/$(resources.inputs.testcluster.name)/kubeconfig --context
          $(resources.inputs.testcluster.name) apply -f /workspace/service.yaml'

To use the cluster resource with Google Kubernetes Engine, you should use thecadata authentication mechanism.

To determine the caData, you can use the following gcloud commands:

gcloud container clusters describe <cluster-name> --format='value(masterAuth.clusterCaCertificate)'

To create a secret with this information, you can use:

CADATA=$(gcloud container clusters describe <cluster-name> --format='value(masterAuth.clusterCaCertificate)')
kubectl create secret generic cluster-ca-data --from-literal=cadata=$CADATA

To retrieve the URL, you can use this gcloud command:

gcloud container clusters describe <cluster-name> --format='value(endpoint)'

Then to use these in a resource, reference the cadata from the secret youcreated above, and use the IP address from the gcloud command as your url(prefixed with https://):

spec:
  type: cluster
  params:
    - name: url
      value: https://<ip address determined above>
  secrets:
    - fieldName: cadata
      secretName: cluster-ca-data
      secretKey: cadata

Storage Resource

The storage resource represents blob storage, that contains either an objector directory. Adding the storage resource as an input to a Task will downloadthe blob and allow the Task to perform the required actions on the contents ofthe blob.

Only blob storage typeGoogle Cloud Storage(gcs) is supported asof now via GCS storage resource andBuildGCS storage resource.

GCS Storage Resource

The gcs storage resource points toGoogle Cloud Storage blob.

To create a GCS type of storage resource using the PipelineResource CRD:

apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
  name: wizzbang-storage
  namespace: default
spec:
  type: storage
  params:
    - name: type
      value: gcs
    - name: location
      value: gs://some-bucket
    - name: dir
      value: "y" # This can have any value to be considered "true"

Params that can be added are the following:

• location: represents the location of the blob storage.
• type: represents the type of blob storage. For GCS storage resource thisvalue should be set to gcs.

• dir: represents whether the blob storage is a directory or not. By defaulta storage artifact is not considered a directory.

  • If the artifact is a directory then -r(recursive) flag is used, tocopy all files under the source directory to a GCS bucket. Eg: gsutil cp -r source_dir/* gs://some-bucket
  • If an artifact is a single file like a zip or tar, then the copy will beonly 1 level deep(not recursive). It will not trigger a copy of subdirectories in the source directory. Eg: gsutil cp source.tar gs://some-bucket.tar. Private buckets can also be configured as storage resources. To access GCSprivate buckets, service accounts with correct permissions are required. Thesecrets field on the storage resource is used for configuring thisinformation. Below is an example on how to create a storage resource with aservice account.

• Refer to theofficial documentationon how to create service accounts and configuringIAM permissionsto access buckets.

• Create a Kubernetes secret from a downloaded service account json key

kubectl create secret generic bucket-sa --from-file=./service_account.json

• To access the GCS private bucket environment variableGOOGLE_APPLICATION_CREDENTIALSshould be set, so apply the above created secret to the GCS storage resourceunder the fieldName key.

apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
  name: wizzbang-storage
  namespace: default
spec:
  type: storage
  params:
    - name: type
      value: gcs
    - name: location
      value: gs://some-private-bucket
    - name: dir
      value: "y"
  secrets:
    - fieldName: GOOGLE_APPLICATION_CREDENTIALS
      secretName: bucket-sa
      secretKey: service_account.json

BuildGCS Storage Resource

The build-gcs storage resource points to aGoogle Cloud Storage blob likeGCS Storage Resource, either in the form of a .ziparchive, or based on the contents of a source manifest file.

In addition to fetching an .zip archive, BuildGCS also unzips it.

ASource Manifest Fileis a JSON object, which is listing other objects in a Cloud Storage that shouldbe fetched. The format of the manifest is a mapping of the destination file pathto the location in a Cloud Storage, where the file’s contents can be found. Thebuild-gcs resource can also do incremental uploads of sources via the SourceManifest File.

To create a build-gcs type of storage resource using the PipelineResourceCRD:

apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
  name: build-gcs-storage
  namespace: default
spec:
  type: storage
  params:
    - name: type
      value: build-gcs
    - name: location
      value: gs://build-crd-tests/rules_docker-master.zip
    - name: artifactType
      value: Archive

Params that can be added are the following:

• location: represents the location of the blob storage.
• type: represents the type of blob storage. For BuildGCS, this value shouldbe set to build-gcs

• artifactType: represent the type of gcs resource. Right now, we supportfollowing types:

• ZipArchive:

  • ZipArchive indicates that the resource fetched is an archive file in thezip format.
  • It unzips the archive and places all the files in the directory, whichis set at runtime.
  • Archive is also supported and is equivalent to ZipArchive, but isdeprecated.
• TarGzArchive:
  • TarGzArchive indicates that the resource fetched is a gzipped archivefile in the tar format.
  • It unzips the archive and places all the files in the directory, whichis set at runtime.
• Manifest:
  • Manifest indicates that the resource should be fetched using a sourcemanifest file.

Private buckets other than the ones accessible by aTaskRun Service Account can not be configuredas storage resources for the build-gcs storage resource right now. This isbecause the container imagegcr.io/cloud-builders//gcs-fetcherdoes not support configuring secrets.

Cloud Event Resource

The cloudevent resource represents acloud event that is sent to a targetURI upon completion of a TaskRun. The cloudevent resource sends Tektonspecific events; the body of the event includes the entire TaskRun spec plusstatus; the types of events defined for now are:

• dev.tekton.event.task.unknown
• dev.tekton.event.task.successful
• dev.tekton.event.task.failed

cloudevent resources are useful to notify a third party upon the completionand status of a TaskRun. In combinations with theTekton triggers project they can be usedto link Task/PipelineRuns asynchronously.

To create a CloudEvent resource using the PipelineResource CRD:

apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
  name: event-to-sink
spec:
  type: cloudEvent
  params:
  - name: targetURI
    value: http://sink:8080

The content of an event is for example:

Context Attributes,
  SpecVersion: 0.2
  Type: dev.tekton.event.task.successful
  Source: /apis/tekton.dev/v1beta1/namespaces/default/taskruns/pipeline-run-api-16aa55-source-to-image-task-rpndl
  ID: pipeline-run-api-16aa55-source-to-image-task-rpndl
  Time: 2019-07-04T11:03:53.058694712Z
  ContentType: application/json
Transport Context,
  URI: /
  Host: my-sink.default.my-cluster.containers.appdomain.cloud
  Method: POST
Data,
  {
    "taskRun": {
      "metadata": {...}
      "spec": {
        "inputs": {...}
        "outputs": {...}
        "serviceAccount": "default",
        "taskRef": {
          "name": "source-to-image",
          "kind": "Task"
        },
        "timeout": "1h0m0s"
      },
      "status": {...}
    }
  }