OLM Integration Bundle Tutorial

The Operator Lifecycle Manager (OLM) is a set of cluster resources that manage the lifecycle of an Operator. The Operator SDK supports both creating manifests for OLM deployment, and testing your Operator on an OLM-enabled Kubernetes cluster.

This document succinctly walks through getting an Operator OLM-ready with bundles, and glosses over explanations of certain steps for brevity. The following documents contain more detail on these steps:

If you are working with package manifests, see the package manifests tutorial once you have completed the Setup section below.

Important: this guide assumes your project was scaffolded with operator-sdk init --project-version=3. These features are unavailable to projects of version 2 or less; this information can be found by inspecting your PROJECT file’s version value.

Setup

Let’s first walk through creating an Operator for memcached, a distributed key-value store.

Follow one of the user guides to develop the memcached-operator in either Go, Ansible, or Helm, depending on which Operator type you are interested in. This guide assumes memcached-operator is on version 0.0.1, which is set in the Makefile variable VERSION.

Enabling OLM

Ensure OLM is enabled on your cluster before following this guide. operator-sdk olm has several subcommands that can install, uninstall, and check the status of particular OLM versions in a cluster.

Note: Certain cluster types may already have OLM enabled, but under a non-default ("olm") namespace, which can be configured by setting --olm-namespace=[non-default-olm-namespace] for operator-sdk olm status|uninstall subcommands.

You can check if OLM is already installed by running the following command, which will detect the installed OLM version automatically (0.15.1 in this example):

  1. $ operator-sdk olm status
  2. INFO[0000] Fetching CRDs for version "0.15.1"
  3. INFO[0002] Fetching resources for version "0.15.1"
  4. INFO[0002] Successfully got OLM status for version "0.15.1"
  6. NAME                                            NAMESPACE    KIND                        STATUS
  7. olm                                                          Namespace                   Installed
  8. operatorgroups.operators.coreos.com                          CustomResourceDefinition    Installed
  9. catalogsources.operators.coreos.com                          CustomResourceDefinition    Installed
  10. subscriptions.operators.coreos.com                           CustomResourceDefinition    Installed
  11. ...

All resources listed should have status Installed.

Note: The operator-sdk olm status command is geared to detect the status of OLM that was installed by installation methods like operator-sdk olm install or by applying OLM manifests directly on the cluster. This command retrieves the resources that were compiled into SDK at the time of installation from the OLM manifests. However, if OLM was installed in a cluster in a custom fashion (such as in OpenShift clusters), it is possible that some resources will show a Not Found status when the operator-sdk olm status command is issued.

To check the true status of such resources in OCP clusters, run:

  1. oc get <resource-name> -n <resource-namespace>

If OLM is not already installed, go ahead and install the latest version:

  1. $ operator-sdk olm install
  2. INFO[0000] Fetching CRDs for version "latest"
  3. INFO[0001] Fetching resources for version "latest"
  4. INFO[0007] Creating CRDs and resources
  5. INFO[0007]   Creating CustomResourceDefinition "clusterserviceversions.operators.coreos.com"
  6. INFO[0007]   Creating CustomResourceDefinition "installplans.operators.coreos.com"
  7. INFO[0007]   Creating CustomResourceDefinition "subscriptions.operators.coreos.com"
  8. ...
  9. NAME                                            NAMESPACE    KIND                        STATUS
  10. clusterserviceversions.operators.coreos.com                  CustomResourceDefinition    Installed
  11. installplans.operators.coreos.com                            CustomResourceDefinition    Installed
  12. subscriptions.operators.coreos.com                           CustomResourceDefinition    Installed
  13. catalogsources.operators.coreos.com                          CustomResourceDefinition    Installed
  14. ...

Note: By default, olm status and olm uninstall auto-detect the OLM version installed in your cluster. This can fail if the installation is broken in some way, so the version of OLM can be overridden using the --version flag provided with these commands.

Creating a bundle

If working with package manifests, see the package manifests tutorial.

We will now create bundle manifests by running make bundle in the root of the memcached-operator project.

  1. $ make bundle
  2. /home/user/go/bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases
  3. operator-sdk generate kustomize manifests -q
  4. kustomize build config/manifests | operator-sdk generate bundle -q --overwrite --version 0.0.1
  5. INFO[0000] Building annotations.yaml
  6. INFO[0000] Writing annotations.yaml in /home/user/go/src/github.com/test-org/memcached-operator/bundle/metadata
  7. INFO[0000] Building Dockerfile
  8. INFO[0000] Writing bundle.Dockerfile in /home/user/go/src/github.com/test-org/memcached-operator
  9. operator-sdk bundle validate ./bundle
  10. INFO[0000] Found annotations file                        bundle-dir=bundle container-tool=docker
  11. INFO[0000] Could not find optional dependencies file     bundle-dir=bundle container-tool=docker
  12. INFO[0000] All validation tests have completed successfully

The above command will have created the following bundle artifacts: a manifests directory (bundle/manifests) containing a CSV and all CRDs from config/crds, metadata directory (bundle/metadata), and bundle.Dockerfile have been created in the Operator project. These files have been statically validated by operator-sdk bundle validate to ensure the on-disk bundle representation is correct.

Deploying an Operator with OLM

At this point in development we’ve generated all files necessary to build the memcached-operator bundle. Now we’re ready to test and deploy the Operator with OLM.

Note: If testing a bundle whose image will be hosted in a registry that is private and/or has a custom CA, these configuration steps must be complete.

Testing bundles

Before proceeding, make sure you’ve Installed OLM onto your cluster.

First, we need to build our bundle. To build a memcached-operator bundle, run:

  1. $ make bundle-build bundle-push BUNDLE_IMG=<some-registry>/memcached-operator-bundle:v0.0.1

Now that the bundle image is present in a registry, operator-sdk run bundle can create a pod to serve that bundle to OLM via a Subscription, along with other OLM objects, ephemerally.

  1. $ operator-sdk run bundle <some-registry>/memcached-operator-bundle:v0.0.1
  2. INFO[0008] Successfully created registry pod: <some-registry>-memcached-operator-bundle-0-0-1
  3. INFO[0008] Created CatalogSource: memcached-operator-catalog
  4. INFO[0008] OperatorGroup "operator-sdk-og" created
  5. INFO[0008] Created Subscription: memcached-operator-v0-0-1-sub
  6. INFO[0019] Approved InstallPlan install-krv7q for the Subscription: memcached-operator-v0-0-1-sub
  7. INFO[0019] Waiting for ClusterServiceVersion "default/memcached-operator.v0.0.1" to reach 'Succeeded' phase
  8. INFO[0019]   Waiting for ClusterServiceVersion "default/memcached-operator.v0.0.1" to appear
  9. INFO[0031]   Found ClusterServiceVersion "default/memcached-operator.v0.0.1" phase: Pending
  10. INFO[0032]   Found ClusterServiceVersion "default/memcached-operator.v0.0.1" phase: Installing
  11. INFO[0040]   Found ClusterServiceVersion "default/memcached-operator.v0.0.1" phase: Succeeded
  12. INFO[0040] OLM has successfully installed "memcached-operator.v0.0.1"

Note: If the bundle that is being installed has dependencies, the --index-image flag allows adding a bundle to a catalog that contains that bundle’s dependencies.

Upgrading a bundle to a newer version

We can use the operator-sdk run bundle-upgrade command with a newer version of bundle image to upgrade an existing operator bundle deployed on cluster. The command automates the manual orchestration typically required to upgrade an operator from one version to another. It extracts the package name from bundle, finds the existing subscription, updates the catalog source, deletes the existing registry pod and creates a new registry pod with the version of bundle image provided in the command.

Let’s upgrade the previously deployed memcached-operator bundle from version 0.0.1 to 0.0.2.

  1. $ operator-sdk run bundle-upgrade <some-registry>/memcached-operator-bundle:v0.0.2
  2. INFO[0002] Found existing subscription with name memcached-operator-bundle-0-0-1-sub and namespace default
  3. INFO[0002] Found existing catalog source with name memcached-operator-catalog and namespace default
  4. INFO[0007] Successfully created registry pod: <some-registry>-memcached-operator-bundle-0-0-2
  5. INFO[0007] Updated catalog source memcached-operator-catalog with address and annotations
  6. INFO[0008] Deleted previous registry pod with name "<some-registry>-memcached-operator-bundle-0-0-1"
  7. INFO[0050] Approved InstallPlan install-c8fkh for the Subscription: memcached-operator-bundle-0-0-1-sub
  8. INFO[0050] Waiting for ClusterServiceVersion "default/memcached-operator.v0.0.2" to reach 'Succeeded' phase
  9. INFO[0050]   Waiting for ClusterServiceVersion "default/memcached-operator.v0.0.2" to appear
  10. INFO[0052]   Found ClusterServiceVersion "default/memcached-operator.v0.0.2" phase: Pending
  11. INFO[0057]   Found ClusterServiceVersion "default/memcached-operator.v0.0.2" phase: InstallReady
  12. INFO[0058]   Found ClusterServiceVersion "default/memcached-operator.v0.0.2" phase: Installing
  13. INFO[0095]   Found ClusterServiceVersion "default/memcached-operator.v0.0.2" phase: Succeeded
  14. INFO[0095] Successfully upgraded to "memcached-operator.v0.0.2"

Note: If a bundle was installed using operator-sdk run bundle with a SQLite index image, the replaces field must be present and populated in the upgraded CSV’s spec.

Upgrading a bundle that was installed traditionally using OLM

An operator bundle can be upgraded even if it was originally deployed using OLM without using the run bundle command.

Let’s see how to deploy an operator bundle traditionally using OLM and then upgrade the operator bundle to a newer version.

First, create a CatalogSource by building the CatalogSource from a catalog.

  1. $ oc create -f catalogsource.yaml
  1. # catalogsource.yaml
  2. apiVersion: operators.coreos.com/v1alpha1
  3. kind: CatalogSource
  4. metadata:
  5.   name: etcdoperator
  6.   namespace: default
  7. spec:
  8.   displayName: Etcd Operators
  9.   image: <some-registry>/etcd-catalog:latest
  10.   sourceType: grpc

Next, install the operator bundle by creating a subscription.

  1. $ oc create -f subscription.yaml
  1. # subscription.yaml
  2. apiVersion: v1
  3. items:
  4. - apiVersion: operators.coreos.com/v1alpha1
  5.   kind: Subscription
  6.   metadata:
  7.     name: etcd
  8.     namespace: default
  9.   spec:
  10.     channel: "stable"
  11.     installPlanApproval: Manual
  12.     name: etcd
  13.     source: etcdoperator
  14.     sourceNamespace: default
  15.     startingCSV: etcdoperator.v0.0.1

Once the Operator bundle is deployed, you can use the run bundle-upgrade command by specifying the new bundle image that you want to upgrade to.

  1. $ operator-sdk run bundle-upgrade <some-registry>/etcd-bundle:v0.0.2
  2. INFO[0000] Found existing subscription with name etcd and namespace default
  3. INFO[0000] Found existing catalog source with name etcdoperator and namespace default
  4. INFO[0005] Successfully created registry pod: <some-registry>-etcd-bundle-0-0-2
  5. INFO[0005] Updated catalog source etcdoperator with address and annotations
  6. INFO[0005] Deleted previous registry pod with name "<some-registry>-etcd-bundle-0-0-1"
  7. INFO[0005] Approved InstallPlan install-6vrzh for the Subscription: etcd
  8. INFO[0005] Waiting for ClusterServiceVersion "default/etcdoperator.v0.0.2" to reach 'Succeeded' phase
  9. INFO[0005]   Waiting for ClusterServiceVersion "default/etcdoperator.v0.0.2" to appear
  10. INFO[0007]   Found ClusterServiceVersion "default/etcdoperator.v0.0.2" phase: Pending
  11. INFO[0008]   Found ClusterServiceVersion "default/etcdoperator.v0.0.2" phase: Installing
  12. INFO[0018]   Found ClusterServiceVersion "default/etcdoperator.v0.0.2" phase: Succeeded
  13. INFO[0018] Successfully upgraded to "etcdoperator.v0.0.2"

Deploying bundles in production

OLM and Operator Registry consumes Operator bundles via a catalog of Operators, implemented as an index image, which are composed of one or more bundles. To build and push a memcached-operator bundle image for version v0.0.1, run:

  1. $ make bundle-build bundle-push BUNDLE_IMG=<some-registry>/memcached-operator-bundle:v0.0.1

Now you can build and push the catalog by running catalog-* Makfile targets, which use the Operator package manager tool opm to build the catalog:

  1. $ make catalog-build catalog-push CATALOG_IMG=<some-registry>/memcached-operator-catalog:v0.0.1

Assuming IMAGE_TAG_BASE = <some-registry>/memcached-operator has the desired tag base, you can inline the above two commands to:

  1. $ make bundle-build bundle-push catalog-build catalog-push

Which will build and push both <some-registry>/memcached-operator-bundle:v0.0.1 and <some-registry>/memcached-operator-catalog:v0.0.1.

Further reading

In-depth discussions of OLM concepts mentioned here:

Last modified April 29, 2022: Docs: Clarification about the use of index image in run bundle(-upgrade) (#5687) (fe978c6d)