OLM Integration User Guide

The Operator Lifecycle Manager (OLM) is a set of cluster resources that manage the lifecycle of an Operator. The Operator SDK supports both readying your Operator project for OLM deployment, and testing your Operator on an OLM-enabled Kubernetes cluster.

Setup

Lets first walk through creating an Operator for memcached, a distributed key-value store.

Follow one of the user guides to develop the memcached-operator in either Go, Ansible, or Helm, depending on which Operator type you are interested in.

Enabling OLM

Ensure OLM is enabled on your cluster before following this guide. operator-sdk olm has several subcommands that can install, uninstall, and check the status of particular OLM versions in a cluster.

Note: Certain cluster types may already have OLM enabled, but under a non-default ("olm") namespace, which can be configured by setting --olm-namespace=[non-default-olm-namespace] for operator-sdk olm subcommands and operator-sdk run packagemanifests.

You can check if OLM is already installed by running the following command, which will detect the installed OLM version automatically (v0.14.1 in this example):

  1. $ operator-sdk olm status
  2. INFO[0000] Fetching CRDs for version "0.14.1"
  3. INFO[0002] Fetching resources for version "0.14.1"
  4. INFO[0002] Successfully got OLM status for version "0.14.1"
  6. NAME                                            NAMESPACE    KIND                        STATUS
  7. olm                                                          Namespace                   Installed
  8. operatorgroups.operators.coreos.com                          CustomResourceDefinition    Installed
  9. catalogsources.operators.coreos.com                          CustomResourceDefinition    Installed
  10. subscriptions.operators.coreos.com                           CustomResourceDefinition    Installed
  11. ...

All resources listed should have status Installed.

If OLM is not already installed, go ahead and install the latest version:

  1. $ operator-sdk olm install
  2. INFO[0000] Fetching CRDs for version "latest"
  3. INFO[0001] Fetching resources for version "latest"
  4. INFO[0007] Creating CRDs and resources
  5. INFO[0007]   Creating CustomResourceDefinition "clusterserviceversions.operators.coreos.com"
  6. INFO[0007]   Creating CustomResourceDefinition "installplans.operators.coreos.com"
  7. INFO[0007]   Creating CustomResourceDefinition "subscriptions.operators.coreos.com"
  8. ...
  9. NAME                                            NAMESPACE    KIND                        STATUS
  10. clusterserviceversions.operators.coreos.com                  CustomResourceDefinition    Installed
  11. installplans.operators.coreos.com                            CustomResourceDefinition    Installed
  12. subscriptions.operators.coreos.com                           CustomResourceDefinition    Installed
  13. catalogsources.operators.coreos.com                          CustomResourceDefinition    Installed
  14. ...

Creating a bundle

Now that we have a working, simple memcached-operator, we can generate manifests and metadata for an Operator bundle. From the bundle docs:

An Operator Bundle is built as a scratch (non-runnable) container image that contains operator manifests and specific metadata in designated directories inside the image. Then, it can be pushed and pulled from an OCI-compliant container registry. Ultimately, an operator bundle will be used by Operator Registry and OLM to install an operator in OLM-enabled clusters.

A bundle consists of on-disk manifests and metadata that define an Operator at a particular version. At this stage in memcached-operator’s development, we only need to worry about generating bundle files; bundle images becomes important once we’re ready to publish our Operator. For a condensed overview of all bundle operations supported by the SDK, read this doc.

We will now create bundle manifests by running operator-sdk generate csv in the root of the memcached-operator project:

Note: while generate csv only officially supports Go Operators, it will generate a barebones CSV for Ansible and Helm Operators that will require manual modification.

  1. $ operator-sdk generate csv --csv-version 0.1.0
  2. INFO[0000] Generating CSV manifest version 0.1.0
  3. INFO[0004] Required csv fields not filled in file deploy/olm-catalog/memcached-operator/manifests/memcached-operator.clusterserviceversion.yaml:
  4.     spec.keywords
  5.     spec.provider
  6. INFO[0004] Created deploy/olm-catalog/memcached-operator/manifests/memcached-operator.clusterserviceversion.yaml

A bundle manifests directory containing a CSV and all CRDs in deploy/crds has been created at deploy/olm-catalog/memcached-operator/manifests:

  1. $ tree deploy/olm-catalog/memcached-operator
  2. deploy/olm-catalog/memcached-operator
  3. └── manifests
  4.     ├── cache.example.com_memcacheds_crd.yaml
  5.     └── memcached-operator.clusterserviceversion.yaml

Next we create bundle metadata and a Dockerfile. Metadata contains information about a particular Operator version available in a registry. OLM uses this information to install specific Operator versions and resolve dependencies.

Of particular note are channels:

Channels allow package authors to write different upgrade paths for different users (e.g. beta vs. stable).

Channels become important when publishing, but we should still be aware of them beforehand as they’re required values in our metadata. bundle create writes the channel stable by default.

Invoking operator-sdk bundle create creates annotations metadata and a bundle.Dockerfile for the memcached-operator:

  1. $ operator-sdk bundle create --generate-only
  2. INFO[0000] Building annotations.yaml                    
  3. INFO[0000] Generating output manifests directory        
  4. INFO[0000] Building Dockerfile

Your deploy/olm-catalog/memcached-operator/metadata/annotations.yaml and bundle.Dockerfile contain the same annotations in slightly different formats. In most cases annotations do not need to be modified; if you do decide to modify them, both sets of annotations must be the same to ensure consistent Operator deployment.

Now that everything need to deploy memcached-operator with OLM has been generated, we want to ensure our bundled metadata and manifests are valid:

  1. $ operator-sdk bundle validate deploy/olm-catalog/memcached-operator/
  2. INFO[0000] All validation tests have completed successfully  bundle-dir=/home/user/projects/memcached-operator/deploy/olm-catalog/memcached-operator

Publishing an Operator

If you eventually wish to publish your Operator, you’ll want to add UI metadata to your CSV. A thorough explanation of which metadata fields are available and what they render as can be found here. Go API markers direct generate csv to automatically and reproducibly populate many of these fields.

Deploying an Operator with OLM

At this point in development we’ve generated all files necessary to build the memcached-operator bundle. Now we’re ready to test and deploy the Operator with OLM.

Testing bundles

Coming soon.

Testing package manifests

operator-sdk run packagemanifests will create an Operator registry from manifests and metadata in the memcached-operator project, and inform OLM that memcached-operator v0.1.0 is ready to be deployed. This process effectively replicates production deployment in a constrained manner to make sure OLM can deploy our Operator successfully before attempting real production deployment.

run packagemanifests performs some optionally configurable setup under the hood, but for most use cases the following invocation is all we need:

  1. $ operator-sdk run packagemanifests --operator-version 0.1.0
  2. INFO[0000] loading Bundles                               dir=deploy/olm-catalog/memcached-operator
  3. INFO[0000] directory                                     dir=deploy/olm-catalog/memcached-operator file=memcached-operator load=bundles
  4. INFO[0000] directory                                     dir=deploy/olm-catalog/memcached-operator file=manifests load=bundles
  5. INFO[0000] found csv, loading bundle                     dir=deploy/olm-catalog/memcached-operator file=memcached-operator.clusterserviceversion.yaml load=bundles
  6. INFO[0000] loading bundle file                           dir=deploy/olm-catalog/memcached-operator/manifests file=example.com_memcacheds_crd.yaml load=bundle
  7. INFO[0000] loading bundle file                           dir=deploy/olm-catalog/memcached-operator/manifests file=memcached-operator.clusterserviceversion.yaml load=bundle
  8. INFO[0000] directory                                     dir=deploy/olm-catalog/memcached-operator file=metadata load=bundles
  9. INFO[0000] loading Packages and Entries                  dir=deploy/olm-catalog/memcached-operator
  10. INFO[0000] directory                                     dir=deploy/olm-catalog/memcached-operator file=memcached-operator load=package
  11. INFO[0000] directory                                     dir=deploy/olm-catalog/memcached-operator file=manifests load=package
  12. INFO[0000] directory                                     dir=deploy/olm-catalog/memcached-operator file=metadata load=package
  13. INFO[0000] Creating registry
  14. INFO[0000]   Creating ConfigMap "olm/memcached-operator-registry-bundles"
  15. INFO[0000]   Creating Deployment "olm/memcached-operator-registry-server"
  16. INFO[0000]   Creating Service "olm/memcached-operator-registry-server"
  17. INFO[0000] Waiting for Deployment "olm/memcached-operator-registry-server" rollout to complete
  18. INFO[0000]   Waiting for Deployment "olm/memcached-operator-registry-server" to rollout: 0 out of 1 new replicas have been updated
  19. INFO[0001]   Waiting for Deployment "olm/memcached-operator-registry-server" to rollout: 0 of 1 updated replicas are available
  20. INFO[0002]   Deployment "olm/memcached-operator-registry-server" successfully rolled out
  21. INFO[0002] Creating resources
  22. INFO[0002]   Creating CatalogSource "default/memcached-operator-ocs"
  23. INFO[0002]   Creating Subscription "default/memcached-operator-v0-0-1-sub"
  24. INFO[0002]   Creating OperatorGroup "default/operator-sdk-og"
  25. INFO[0002] Waiting for ClusterServiceVersion "default/memcached-operator.v0.1.0" to reach 'Succeeded' phase
  26. INFO[0002]   Waiting for ClusterServiceVersion "default/memcached-operator.v0.1.0" to appear
  27. INFO[0034]   Found ClusterServiceVersion "default/memcached-operator.v0.1.0" phase: Pending
  28. INFO[0035]   Found ClusterServiceVersion "default/memcached-operator.v0.1.0" phase: InstallReady
  29. INFO[0036]   Found ClusterServiceVersion "default/memcached-operator.v0.1.0" phase: Installing
  30. INFO[0036]   Found ClusterServiceVersion "default/memcached-operator.v0.1.0" phase: Succeeded
  31. INFO[0037] Successfully installed "memcached-operator.v0.1.0" on OLM version "0.14.1"
  33. NAME                            NAMESPACE    KIND                        STATUS
  34. memcacheds.cache.example.com    default      CustomResourceDefinition    Installed
  35. memcached-operator.v0.1.0       default      ClusterServiceVersion       Installed

As long as both the ClusterServiceVersion and all CustomResourceDefinition‘s return an Installed status, the memcached-operator has been deployed successfully.

Now that we’re done testing the memcached-operator, we should probably clean up the Operator’s resources. The operator-sdk cleanup packagemanifests command will do this for you:

  1. $ operator-sdk cleanup packagemanifests --operator-version 0.1.0
  2. INFO[0000] loading Bundles                               dir=deploy/olm-catalog/memcached-operator
  3. INFO[0000] directory                                     dir=deploy/olm-catalog/memcached-operator file=memcached-operator load=bundles
  4. INFO[0000] directory                                     dir=deploy/olm-catalog/memcached-operator file=manifests load=bundles
  5. INFO[0000] found csv, loading bundle                     dir=deploy/olm-catalog/memcached-operator file=memcached-operator.clusterserviceversion.yaml load=bundles
  6. INFO[0000] loading bundle file                           dir=deploy/olm-catalog/memcached-operator/manifests file=example.com_memcacheds_crd.yaml load=bundle
  7. INFO[0000] loading bundle file                           dir=deploy/olm-catalog/memcached-operator/manifests file=memcached-operator.clusterserviceversion.yaml load=bundle
  8. INFO[0000] directory                                     dir=deploy/olm-catalog/memcached-operator file=metadata load=bundles
  9. INFO[0000] loading Packages and Entries                  dir=deploy/olm-catalog/memcached-operator
  10. INFO[0000] directory                                     dir=deploy/olm-catalog/memcached-operator file=memcached-operator load=package
  11. INFO[0000] directory                                     dir=deploy/olm-catalog/memcached-operator file=manifests load=package
  12. INFO[0000] directory                                     dir=deploy/olm-catalog/memcached-operator file=metadata load=package
  13. INFO[0000] Deleting resources
  14. INFO[0000]   Deleting CatalogSource "default/memcached-operator-ocs"
  15. INFO[0000]   Deleting Subscription "default/memcached-operator-v0-0-1-sub"
  16. INFO[0000]   Deleting OperatorGroup "default/operator-sdk-og"
  17. INFO[0000]   Deleting CustomResourceDefinition "default/memcacheds.example.com"
  18. INFO[0000]   Deleting ClusterServiceVersion "default/memcached-operator.v0.1.0"
  19. INFO[0000]   Waiting for deleted resources to disappear
  20. INFO[0001] Successfully uninstalled "memcached-operator.v0.1.0" on OLM version "0.14.1"

Production bundle deployment

OLM and Operator Registry consumes Operator bundles via an index image, which are composed of one or more bundles. To build a memcached-operator bundle, run:

  1. $ docker build -f bundle.Dockerfile -t quay.io/<username>/memcached-operator:v0.1.0 .

Although we’ve validated on-disk manifests and metadata, we also must make sure the bundle itself is valid:

  1. $ docker push quay.io/<username>/memcached-operator:v0.1.0
  2. $ operator-sdk bundle validate quay.io/<username>/memcached-operator:v0.1.0
  3. INFO[0000] Unpacked image layers                         bundle-dir=/tmp/bundle-716785960 container-tool=docker
  4. INFO[0000] running docker pull                           bundle-dir=/tmp/bundle-716785960 container-tool=docker
  5. INFO[0002] running docker save                           bundle-dir=/tmp/bundle-716785960 container-tool=docker
  6. INFO[0002] All validation tests have completed successfully  bundle-dir=/tmp/bundle-716785960 container-tool=docker

Currently the SDK does not build index images; instead, use the Operator package manager tool opm to manage index images. Once one has been built, follow the OLM docs on adding an index to a cluster registry.

Last modified May 22, 2020: cleanup: remove `—olm` from docs, deprecate `—kubeconfig` (#3074) (18de001b)