Working with bundle images

You can use the Operator SDK to package, deploy, and upgrade Operators in the bundle format for use on Operator Lifecycle Manager (OLM).

Bundling an Operator

The Operator bundle format is the default packaging method for Operator SDK and Operator Lifecycle Manager (OLM). You can get your Operator ready for use on OLM by using the Operator SDK to build and push your Operator project as a bundle image.

Prerequisites

• Operator SDK CLI installed on a development workstation

• OpenShift CLI (oc) v4.13+ installed

• Operator project initialized by using the Operator SDK

• If your Operator is Go-based, your project must be updated to use supported images for running on OKD

Procedure

1. Run the following make commands in your Operator project directory to build and push your Operator image. Modify the IMG argument in the following steps to reference a repository that you have access to. You can obtain an account for storing containers at repository sites such as Quay.io.

   1. Build the image:

      $ make docker-build IMG=<registry>/<user>/<operator_image_name>:<tag>

      The Dockerfile generated by the SDK for the Operator explicitly references GOARCH=amd64 for go build. This can be amended to GOARCH=$TARGETARCH for non-AMD64 architectures. Docker will automatically set the environment variable to the value specified by –platform. With Buildah, the –build-arg will need to be used for the purpose. For more information, see Multiple Architectures.

   2. Push the image to a repository:

      $ make docker-push IMG=<registry>/<user>/<operator_image_name>:<tag>

2. Create your Operator bundle manifest by running the make bundle command, which invokes several commands, including the Operator SDK generate bundle and bundle validate subcommands:

   $ make bundle IMG=<registry>/<user>/<operator_image_name>:<tag>

   Bundle manifests for an Operator describe how to display, create, and manage an application. The make bundle command creates the following files and directories in your Operator project:

   • A bundle manifests directory named bundle/manifests that contains a ClusterServiceVersion object

   • A bundle metadata directory named bundle/metadata

   • All custom resource definitions (CRDs) in a config/crd directory

   • A Dockerfile bundle.Dockerfile

   These files are then automatically validated by using operator-sdk bundle validate to ensure the on-disk bundle representation is correct.

3. Build and push your bundle image by running the following commands. OLM consumes Operator bundles using an index image, which reference one or more bundle images.

   1. Build the bundle image. Set BUNDLE_IMG with the details for the registry, user namespace, and image tag where you intend to push the image:

      $ make bundle-build BUNDLE_IMG=<registry>/<user>/<bundle_image_name>:<tag>

   2. Push the bundle image:

      $ docker push <registry>/<user>/<bundle_image_name>:<tag>

Deploying an Operator with Operator Lifecycle Manager

Operator Lifecycle Manager (OLM) helps you to install, update, and manage the lifecycle of Operators and their associated services on a Kubernetes cluster. OLM is installed by default on OKD and runs as a Kubernetes extension so that you can use the web console and the OpenShift CLI (oc) for all Operator lifecycle management functions without any additional tools.

The Operator bundle format is the default packaging method for Operator SDK and OLM. You can use the Operator SDK to quickly run a bundle image on OLM to ensure that it runs properly.

Prerequisites

• Operator SDK CLI installed on a development workstation

• Operator bundle image built and pushed to a registry

• OLM installed on a Kubernetes-based cluster (v1.16.0 or later if you use apiextensions.k8s.io/v1 CRDs, for example OKD 4.13)

• Logged in to the cluster with oc using an account with cluster-admin permissions

• If your Operator is Go-based, your project must be updated to use supported images for running on OKD

Procedure

1. Enter the following command to run the Operator on the cluster:

   $ operator-sdk run bundle \(1)
       -n <namespace> \(2)
       <registry>/<user>/<bundle_image_name>:<tag> (3)

   1	The run bundle command creates a valid file-based catalog and installs the Operator bundle on your cluster using OLM.
   2	Optional: By default, the command installs the Operator in the currently active project in your ~/.kube/config file. You can add the -n flag to set a different namespace scope for the installation.
   3	If you do not specify an image, the command uses quay.io/operator-framework/opm:latest as the default index image. If you specify an image, the command uses the bundle image itself as the index image.

   As of OKD 4.11, the run bundle command supports the file-based catalog format for Operator catalogs by default. The deprecated SQLite database format for Operator catalogs continues to be supported; however, it will be removed in a future release. It is recommended that Operator authors migrate their workflows to the file-based catalog format.

   This command performs the following actions:

   • Create an index image referencing your bundle image. The index image is opaque and ephemeral, but accurately reflects how a bundle would be added to a catalog in production.

   • Create a catalog source that points to your new index image, which enables OperatorHub to discover your Operator.

   • Deploy your Operator to your cluster by creating an OperatorGroup, Subscription, InstallPlan, and all other required resources, including RBAC.

Additional resources

• File-based catalogs in Operator Framework packaging format

• File-based catalogs in Managing custom catalogs

• Bundle format

Publishing a catalog containing a bundled Operator

To install and manage Operators, Operator Lifecycle Manager (OLM) requires that Operator bundles are listed in an index image, which is referenced by a catalog on the cluster. As an Operator author, you can use the Operator SDK to create an index containing the bundle for your Operator and all of its dependencies. This is useful for testing on remote clusters and publishing to container registries.

Prerequisites

• Operator SDK CLI installed on a development workstation

• Operator bundle image built and pushed to a registry

• OLM installed on a Kubernetes-based cluster (v1.16.0 or later if you use apiextensions.k8s.io/v1 CRDs, for example OKD 4.13)

• Logged in to the cluster with oc using an account with cluster-admin permissions

Procedure

1. Run the following make command in your Operator project directory to build an index image containing your Operator bundle:

   $ make catalog-build CATALOG_IMG=<registry>/<user>/<index_image_name>:<tag>

   where the CATALOG_IMG argument references a repository that you have access to. You can obtain an account for storing containers at repository sites such as Quay.io.

2. Push the built index image to a repository:

   $ make catalog-push CATALOG_IMG=<registry>/<user>/<index_image_name>:<tag>

   You can use Operator SDK make commands together if you would rather perform multiple actions in sequence at once. For example, if you had not yet built a bundle image for your Operator project, you can build and push both a bundle image and an index image with the following syntax: $ make bundle-build bundle-push catalog-build catalog-push \ BUNDLE_IMG=<bundle_image_pull_spec> \ CATALOG_IMG=<index_image_pull_spec> Alternatively, you can set the IMAGE_TAG_BASE field in your Makefile to an existing repository: IMAGE_TAG_BASE=quay.io/example/my-operator You can then use the following syntax to build and push images with automatically-generated names, such as quay.io/example/my-operator-bundle:v0.0.1 for the bundle image and quay.io/example/my-operator-catalog:v0.0.1 for the index image: $ make bundle-build bundle-push catalog-build catalog-push

3. Define a CatalogSource object that references the index image you just generated, and then create the object by using the oc apply command or web console:

   Example CatalogSource YAML

   apiVersion: operators.coreos.com/v1alpha1
   kind: CatalogSource
   metadata:
     name: cs-memcached
     namespace: default
   spec:
     displayName: My Test
     publisher: Company
     sourceType: grpc
     grpcPodConfig:
       securityContextConfig: <security_mode> (1)
     image: quay.io/example/memcached-catalog:v0.0.1 (2)
     updateStrategy:
       registryPoll:
         interval: 10m

   1	Specify the value of legacy or restricted. If the field is not set, the default value is legacy. In a future OKD release, it is planned that the default value will be restricted. If your catalog cannot run with restricted permissions, it is recommended that you manually set this field to legacy.
   2	Set image to the image pull spec you used previously with the CATALOG_IMG argument.

4. Check the catalog source:

   $ oc get catalogsource

   Example output

   NAME           DISPLAY     TYPE   PUBLISHER   AGE
   cs-memcached   My Test     grpc   Company     4h31m

Verification

1. Install the Operator using your catalog:

   1. Define an OperatorGroup object and create it by using the oc apply command or web console:

      Example OperatorGroup YAML

      apiVersion: operators.coreos.com/v1
      kind: OperatorGroup
      metadata:
        name: my-test
        namespace: default
      spec:
        targetNamespaces:
        - default

   2. Define a Subscription object and create it by using the oc apply command or web console:

      Example Subscription YAML

      apiVersion: operators.coreos.com/v1alpha1
      kind: Subscription
      metadata:
        name: catalogtest
        namespace: default
      spec:
        channel: "alpha"
        installPlanApproval: Manual
        name: catalog
        source: cs-memcached
        sourceNamespace: default
        startingCSV: memcached-operator.v0.0.1

2. Verify the installed Operator is running:

   1. Check the Operator group:

      $ oc get og

      Example output

      NAME             AGE
      my-test           4h40m

   2. Check the cluster service version (CSV):

      $ oc get csv

      Example output

      NAME                        DISPLAY   VERSION   REPLACES   PHASE
      memcached-operator.v0.0.1   Test      0.0.1                Succeeded

   3. Check the pods for the Operator:

      $ oc get pods

      Example output

      NAME                                                              READY   STATUS      RESTARTS   AGE
      9098d908802769fbde8bd45255e69710a9f8420a8f3d814abe88b68f8ervdj6   0/1     Completed   0          4h33m
      catalog-controller-manager-7fd5b7b987-69s4n                       2/2     Running     0          4h32m
      cs-memcached-7622r                                                1/1     Running     0          4h33m

Additional resources

• See Managing custom catalogs for details on direct usage of the opm CLI for more advanced use cases.

Testing an Operator upgrade on Operator Lifecycle Manager

You can quickly test upgrading your Operator by using Operator Lifecycle Manager (OLM) integration in the Operator SDK, without requiring you to manually manage index images and catalog sources.

The run bundle-upgrade subcommand automates triggering an installed Operator to upgrade to a later version by specifying a bundle image for the later version.

Prerequisites

• Operator installed with OLM either by using the run bundle subcommand or with traditional OLM installation

• A bundle image that represents a later version of the installed Operator

Procedure

1. If your Operator has not already been installed with OLM, install the earlier version either by using the run bundle subcommand or with traditional OLM installation.

   If the earlier version of the bundle was installed traditionally using OLM, the newer bundle that you intend to upgrade to must not exist in the index image referenced by the catalog source. Otherwise, running the run bundle-upgrade subcommand will cause the registry pod to fail because the newer bundle is already referenced by the index that provides the package and cluster service version (CSV).

   For example, you can use the following run bundle subcommand for a Memcached Operator by specifying the earlier bundle image:

   $ operator-sdk run bundle <registry>/<user>/memcached-operator:v0.0.1

   Example output

   INFO[0006] Creating a File-Based Catalog of the bundle "quay.io/demo/memcached-operator:v0.0.1"
   INFO[0008] Generated a valid File-Based Catalog
   INFO[0012] Created registry pod: quay-io-demo-memcached-operator-v1-0-1
   INFO[0012] Created CatalogSource: memcached-operator-catalog
   INFO[0012] OperatorGroup "operator-sdk-og" created
   INFO[0012] Created Subscription: memcached-operator-v0-0-1-sub
   INFO[0015] Approved InstallPlan install-h9666 for the Subscription: memcached-operator-v0-0-1-sub
   INFO[0015] Waiting for ClusterServiceVersion "my-project/memcached-operator.v0.0.1" to reach 'Succeeded' phase
   INFO[0015] Waiting for ClusterServiceVersion ""my-project/memcached-operator.v0.0.1" to appear
   INFO[0026] Found ClusterServiceVersion "my-project/memcached-operator.v0.0.1" phase: Pending
   INFO[0028] Found ClusterServiceVersion "my-project/memcached-operator.v0.0.1" phase: Installing
   INFO[0059] Found ClusterServiceVersion "my-project/memcached-operator.v0.0.1" phase: Succeeded
   INFO[0059] OLM has successfully installed "memcached-operator.v0.0.1"

2. Upgrade the installed Operator by specifying the bundle image for the later Operator version:

   $ operator-sdk run bundle-upgrade <registry>/<user>/memcached-operator:v0.0.2

   Example output

   INFO[0002] Found existing subscription with name memcached-operator-v0-0-1-sub and namespace my-project
   INFO[0002] Found existing catalog source with name memcached-operator-catalog and namespace my-project
   INFO[0008] Generated a valid Upgraded File-Based Catalog
   INFO[0009] Created registry pod: quay-io-demo-memcached-operator-v0-0-2
   INFO[0009] Updated catalog source memcached-operator-catalog with address and annotations
   INFO[0010] Deleted previous registry pod with name "quay-io-demo-memcached-operator-v0-0-1"
   INFO[0041] Approved InstallPlan install-gvcjh for the Subscription: memcached-operator-v0-0-1-sub
   INFO[0042] Waiting for ClusterServiceVersion "my-project/memcached-operator.v0.0.2" to reach 'Succeeded' phase
   INFO[0019] Found ClusterServiceVersion "my-project/memcached-operator.v0.0.2" phase: Pending
   INFO[0042] Found ClusterServiceVersion "my-project/memcached-operator.v0.0.2" phase: InstallReady
   INFO[0043] Found ClusterServiceVersion "my-project/memcached-operator.v0.0.2" phase: Installing
   INFO[0044] Found ClusterServiceVersion "my-project/memcached-operator.v0.0.2" phase: Succeeded
   INFO[0044] Successfully upgraded to "memcached-operator.v0.0.2"

3. Clean up the installed Operators:

   $ operator-sdk cleanup memcached-operator

Additional resources

• Traditional Operator installation with OLM

Controlling Operator compatibility with OKD versions

When an API is removed from an OKD version, Operators running on that cluster version that are still using removed APIs will no longer work properly. As an Operator author, you should plan to update your Operator projects to accommodate API deprecation and removal to avoid interruptions for users of your Operator.

If a cluster administrator has installed your Operator, before they upgrade to the next version of OKD, they must ensure a version of your Operator is installed that is compatible with that next cluster version. While it is recommended that you update your Operator projects to no longer use deprecated or removed APIs, if you still need to publish your Operator bundles with removed APIs for continued use on earlier versions of OKD, ensure that the bundle is configured accordingly.

The following procedure helps prevent administrators from installing versions of your Operator on an incompatible version of OKD. These steps also prevent administrators from upgrading to a newer version of OKD that is incompatible with the version of your Operator that is currently installed on their cluster.

This procedure is also useful when you know that the current version of your Operator will not work well, for any reason, on a specific OKD version. By defining the cluster versions where the Operator should be distributed, you ensure that the Operator does not appear in a catalog of a cluster version which is outside of the allowed range.

Prerequisites

• An existing Operator project

Procedure

1. If you know that a specific bundle of your Operator is not supported and will not work correctly on OKD later than a certain cluster version, configure the maximum version of OKD that your Operator is compatible with. In your Operator project’s cluster service version (CSV), set the olm.maxOpenShiftVersion annotation to prevent administrators from upgrading their cluster before upgrading the installed Operator to a compatible version:

   You must use olm.maxOpenShiftVersion annotation only if your Operator bundle version cannot work in later versions. Be aware that cluster admins cannot upgrade their clusters with your solution installed. If you do not provide later version and a valid upgrade path, cluster admins may uninstall your Operator and can upgrade the cluster version.

   Example CSV with olm.maxOpenShiftVersion annotation

   apiVersion: operators.coreos.com/v1alpha1
   kind: ClusterServiceVersion
   metadata:
     annotations:
       "olm.properties": '[{"type": "olm.maxOpenShiftVersion", "value": "<cluster_version>"}]' (1)

   1	Specify the maximum cluster version of OKD that your Operator is compatible with. For example, setting value to 4.9 prevents cluster upgrades to OKD versions later than 4.9 when this bundle is installed on a cluster.

2. If your bundle is intended for distribution in a Red Hat-provided Operator catalog, configure the compatible versions of OKD for your Operator by setting the following properties. This configuration ensures your Operator is only included in catalogs that target compatible versions of OKD:

   This step is only valid when publishing Operators in Red Hat-provided catalogs. If your bundle is only intended for distribution in a custom catalog, you can skip this step. For more details, see “Red Hat-provided Operator catalogs”.

   1. Set the com.redhat.openshift.versions annotation in your project’s bundle/metadata/annotations.yaml file:

      Example bundle/metadata/annotations.yaml file with compatible versions

      com.redhat.openshift.versions: "v4.7-v4.9" (1)

      1	Set to a range or single version.

   2. To prevent your bundle from being carried on to an incompatible version of OKD, ensure that the index image is generated with the proper com.redhat.openshift.versions label in your Operator’s bundle image. For example, if your project was generated using the Operator SDK, update the bundle.Dockerfile file:

      Example bundle.Dockerfile with compatible versions

      LABEL com.redhat.openshift.versions="<versions>" (1)

      1	Set to a range or single version, for example, v4.7-v4.9. This setting defines the cluster versions where the Operator should be distributed, and the Operator does not appear in a catalog of a cluster version which is outside of the range.

You can now bundle a new version of your Operator and publish the updated version to a catalog for distribution.

Additional resources

• Managing OpenShift Versions in the Certified Operator Build Guide

• Updating installed Operators

• Red Hat-provided Operator catalogs

Additional resources

• See Operator Framework packaging format for details on the bundle format.

• See Managing custom catalogs for details on adding bundle images to index images by using the opm command.

• See Operator Lifecycle Manager workflow for details on how upgrades work for installed Operators.