Ansible Operator Advanced Options

This document shows the advanced options available to a developer of an ansible operator.

Runner Directory

The ansible runner will keep information about the ansible run in the container. This is located /tmp/ansible-operator/runner/<group>/<version>/<kind>/<namespace>/<name>. To learn more about the runner directory you can read the ansible-runner docs.

Owner Reference Injection

Owner references enable Kubernetes Garbage Collection to clean up after a CR is deleted. Owner references are injected by ansible operators by default by the proxy.

Owner references only apply to resources in the same namespace as the CR. Resources outside the namespace of the CR will automatically be annotated with operator-sdk/primary-resource and operator-sdk/primary-resource-type to track creation. These resources will not be automatically garbage collected. To handle deletion of these resources, use a finalizer.

You may want to manage what your operator watches and the owner references. This means that your operator will need to understand how to clean up after itself when your CR is deleted. To disable these features you will need to edit your build/Dockerfile to include the line below.

NOTE: That if you use this feature there will be a warning that dependent watches is turned off but there will be no error. WARNING: Once a CR is deployed without owner reference injection, there is no automatic way to add those references.

  1. ENTRYPOINT ["/usr/local/bin/entrypoint", "--inject-owner-ref=false"]

If you have created resources without owner reference injection, it is possible to manually to update resources following this guide.

Max Workers

Increasing the number of workers allows events to be processed concurrently, which can improve reconciliation performance.

Worker maximums can be set in two ways. Operator authors and admins can set the max workers default by including extra args to the operator container in deploy/operator.yaml. (Otherwise, the default is 1 worker.)

NOTE: Admins using OLM should use the environment variable instead of the extra args.

  1. - name: operator
  2.   image: "quay.io/asmacdo/memcached-operator:v0.0.0"
  3.   imagePullPolicy: "Always"
  4.   args:
  5.     - "--max-workers"
  6.     - "3"

Operator admins can override the value by setting an environment variable in the format WORKER_<kind>_<group>. This variable must be all uppercase, and periods (e.g. in the group name) are replaced with underscores.

For the memcached operator example, the component parts are retrieved with a GET on the operator:

  1. $ kubectl get memcacheds example-memcached -o yaml
  3. apiVersion: cache.example.com/v1alpha1
  4. kind: Memcached
  5. metadata:
  6.   name: example-memcached
  7.   namespace: default

From this data, we can see that the environment variable will be WORKER_MEMCACHED_CACHE_EXAMPLE_COM, which we can then add to deploy/operator.yaml:

  1. - name: operator
  2.   image: "quay.io/asmacdo/memcached-operator:v0.0.0"
  3.   imagePullPolicy: "Always"
  4.   args:
  5.     # This default is overridden.
  6.     - "--max-workers"
  7.     - "3"
  8.   env:
  9.     # This value is used
  10.     - name: WORKER_MEMCACHED_CACHE_EXAMPLE_COM
  11.       value: "6"

Ansible Verbosity

Setting the verbosity at which ansible-runner is run controls how verbose the output of ansible-playbook will be. The normal rules for verbosity apply here, where higher values mean more output. Acceptable values range from 0 (only the most severe messages are output) to 7 (all debugging messages are output).

There are three ways to configure the verbosity argument to the ansible-runner command:

  1. Operator authors and admins can set the Ansible verbosity by including extra args to the operator container in the operator deployment.
  2. Operator admins can set Ansible verbosity by setting an environment variable in the format ANSIBLE_VERBOSITY_<kind>_<group>. This variable must be all uppercase and all periods (e.g. in the group name) are replaced with underscore.
  3. Operator users, authors, and admins can set the Ansible verbosity by setting the "ansible.operator-sdk/verbosity" annotation on the Custom Resource.

Examples

For demonstration purposes, let us assume that we have a database operator that supports two Kinds – MongoDB and PostgreSQL – in the db.example.com Group. We have only recently implemented the support for the MongoDB Kind so we want reconciles for this Kind to be more verbose. Our operator container’s spec in our deploy/operator.yaml might look something like:

  1. - name: operator
  2.   image: "quay.io/example/database-operator:v1.0.0"
  3.   imagePullPolicy: "Always"
  4.   args:
  5.     # This value applies to all GVKs specified in watches.yaml
  6.     # that are not overriden by environment variables.
  7.     - "--ansible-verbosity"
  8.     - "1"
  9.   env:
  10.     # Override the verbosity for the MongoDB kind
  11.     - name: ANSIBLE_VERBOSITY_MONGODB_DB_EXAMPLE_COM
  12.       value: "4"

Once the Operator is deployed, the only way to change the verbosity is via the "ansible.operator-sdk/verbosity" annotation. Continuing with our example, our CR may look like:

  1. apiVersion: "db.example.com/v1"
  2. kind: "PostgreSQL"
  3. metadata:
  4.   name: "example-db"
  5.   annotations:
  6.     "ansible.operator-sdk/verbosity": "5"
  7. spec: {}

Custom Resources with OpenApi Validation

Currently, SDK tool does not support and will not generate automatically the CRD’s using the OpenAPI spec to perform validations.

However, it can be done manually by adding its validations as you can check in the following example.

Example

  1. apiVersion: apiextensions.k8s.io/v1beta1
  2. kind: CustomResourceDefinition
  3. metadata:
  4.   name: memcacheds.cache.example.com
  5. spec:
  6.   group: cache.example.com
  7.   names:
  8.     kind: Memcached
  9.     listKind: MemcachedList
  10.     plural: memcacheds
  11.     singular: memcached
  12.   scope: Namespaced
  13.   subresources:
  14.     status: {}
  15.   validation:
  16.     openAPIV3Schema:
  17.       description: Memcached is the Schema for the memcacheds API
  18.       properties:
  19.         apiVersion:
  20.           description: 'APIVersion defines the versioned schema of this representation
  21.             of an object. Servers should convert recognized schemas to the latest
  22.             internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
  23.           type: string
  24.         kind:
  25.           description: 'Kind is a string value representing the REST resource this
  26.             object represents. Servers may infer this from the endpoint the client
  27.             submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
  28.           type: string
  29.         metadata:
  30.           type: object
  31.         spec:
  32.           description: MemcachedSpec defines the desired state of Memcached
  33.           properties:
  34.             size:
  35.               description: Size is the size of the memcached deployment
  36.               format: int32
  37.               type: integer
  38.           required:
  39.           - size
  40.           type: object
  41.         status:
  42.           description: MemcachedStatus defines the observed state of Memcached
  43.           properties:
  44.             nodes:
  45.               description: Nodes are the names of the memcached pods
  46.               items:
  47.                 type: string
  48.               type: array
  49.           required:
  50.           - nodes
  51.           type: object
  52.       type: object
  53.   versions:
  54.   - name: v1alpha1
  55.     served: true
  56.     storage: true

Last modified June 11, 2020: Fix links after kubernetes renamed access-kubernetes to extend-kubernetes-api (#3219) (ee8ff590)