Ansible Operator Quickstart

This guide walks through an example of building a simple memcached-operator powered by Ansible using tools and libraries provided by the Operator SDK.

Create a new project

After installing the Operator SDK CLI and ansible operator prerequisites, use the CLI to create a new Ansible-based memcached-operator project:

  1. $ operator-sdk new memcached-operator --api-version=cache.example.com/v1alpha1 --kind=Memcached --type=ansible
  2. $ cd memcached-operator

This creates the memcached-operator project specifically for watching the Memcached resource with APIVersion cache.example.com/v1apha1 and Kind Memcached.

To learn more about the project directory structure, see project layout doc.

Operator scope

Read the operator scope documentation on how to run your operator as namespace-scoped vs cluster-scoped.

Customize the operator logic

For this example the memcached-operator will execute the following reconciliation logic for each Memcached Custom Resource (CR):

  • Create a memcached Deployment if it doesn’t exist
  • Ensure that the Deployment size is the same as specified by the Memcached CR

Watch the Memcached CR

By default, the memcached-operator watches Memcached resource events as shown in watches.yaml and executes Ansible Role Memcached:

  1. ---
  2. - version: v1alpha1
  3.   group: cache.example.com
  4.   kind: Memcached

Options

Role Specifying a role option in watches.yaml will configure the operator to use this specified path when launching ansible-runner with an Ansible Role. By default, the new command will fill in an absolute path to where your role should go.

  1. ---
  2. - version: v1alpha1
  3.   group: cache.example.com
  4.   kind: Memcached
  5.   role: memcached

Playbook Specifying a playbook option in watches.yaml will configure the operator to use this specified path when launching ansible-runner with an Ansible Playbook

  1. ---
  2. - version: v1alpha1
  3.   group: cache.example.com
  4.   kind: Memcached
  5.   playbook: playbook.yaml

See watches reference for more information.

Building the Memcached Ansible Role

The first thing to do is to modify the generated Ansible role under roles/memcached. This Ansible Role controls the logic that is executed when a resource is modified.

Define the Memcached spec

Defining the spec for an Ansible Operator can be done entirely in Ansible. The Ansible Operator will simply pass all key value pairs listed in the Custom Resource spec field along to Ansible as extra variables. The names of all variables in the spec field are converted to snake_case by the operator before running ansible. For example, serviceAccount in the spec becomes service_account in ansible. It is recommended that you perform some type validation in Ansible on the variables to ensure that your application is receiving expected input.

First, set a default in case the user doesn’t set the spec field by modifying roles/memcached/defaults/main.yml:

  1. size: 1

Defining the Memcached deployment

Now that we have the spec defined, we can define what Ansible is actually executed on resource changes. Since this is an Ansible Role, the default behavior will be to execute the tasks in roles/memcached/tasks/main.yml. We want Ansible to create a deployment if it does not exist which runs the memcached:1.4.36-alpine image. Ansible 2.5+ supports the k8s Ansible Module which we will leverage to control the deployment definition.

Modify roles/memcached/tasks/main.yml to look like the following:

  1. ---
  2. - name: start memcached
  3.   community.kubernetes.k8s:
  4.     definition:
  5.       kind: Deployment
  6.       apiVersion: apps/v1
  7.       metadata:
  8.         name: '{{ meta.name }}-memcached'
  9.         namespace: '{{ meta.namespace }}'
  10.       spec:
  11.         replicas: "{{size}}"
  12.         selector:
  13.           matchLabels:
  14.             app: memcached
  15.         template:
  16.           metadata:
  17.             labels:
  18.               app: memcached
  19.           spec:
  20.             containers:
  21.             - name: memcached
  22.               command:
  23.               - memcached
  24.               - -m=64
  25.               - -o
  26.               - modern
  27.               - -v
  28.               image: "docker.io/memcached:1.4.36-alpine"
  29.               ports:
  30.                 - containerPort: 11211

It is important to note that we used the size variable to control how many replicas of the Memcached deployment we want. We set the default to 1, but any user can create a Custom Resource that overwrites the default.

Build and run the operator

Before running the operator, Kubernetes needs to know about the new custom resource definition the operator will be watching.

Deploy the CRD:

  1. $ kubectl create -f deploy/crds/cache.example.com_memcacheds_crd.yaml

Once this is done, there are two ways to run the operator:

  • As a pod inside a Kubernetes cluster
  • As a go program outside the cluster using operator-sdk

1. Run as a pod inside a Kubernetes cluster

Running as a pod inside a Kubernetes cluster is preferred for production use.

Build the memcached-operator image and push it to a registry:

  1. $ operator-sdk build quay.io/example/memcached-operator:v0.0.1
  2. $ docker push quay.io/example/memcached-operator:v0.0.1

Kubernetes deployment manifests are generated in deploy/operator.yaml. The deployment image in this file needs to be modified from the placeholder REPLACE_IMAGE to the previous built image. To do this run:

  1. $ sed -i 's|REPLACE_IMAGE|quay.io/example/memcached-operator:v0.0.1|g' deploy/operator.yaml

Note If you are performing these steps on OSX, use the following sed commands instead:

  1. $ sed -i "" 's|REPLACE_IMAGE|quay.io/example/memcached-operator:v0.0.1|g' deploy/operator.yaml

Deploy the memcached-operator:

  1. $ kubectl create -f deploy/service_account.yaml
  2. $ kubectl create -f deploy/role.yaml
  3. $ kubectl create -f deploy/role_binding.yaml
  4. $ kubectl create -f deploy/operator.yaml

Verify that the memcached-operator is up and running:

  1. $ kubectl get deployment
  2. NAME                     DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
  3. memcached-operator       1         1         1            1           1m

2. Run outside the cluster

This method is preferred during the development cycle to speed up deployment and testing.

Note: Ensure that Ansible Runner and Ansible Runner HTTP Plugin is installed or else you will see unexpected errors from Ansible Runner when a Custom Resource is created.

It is also important that the role path referenced in watches.yaml exists on your machine. Since we are normally used to using a container where the Role is put on disk for us, we need to manually copy our role to the configured Ansible Roles path (e.g /etc/ansible/roles.

Run the operator locally with the default Kubernetes config file present at $HOME/.kube/config:

  1. $ operator-sdk run local
  2. INFO[0000] Go Version: go1.10
  3. INFO[0000] Go OS/Arch: darwin/amd64
  4. INFO[0000] operator-sdk Version: 0.0.5+git

Run the operator locally with a provided Kubernetes config file:

  1. $ operator-sdk run local --kubeconfig=config
  2. INFO[0000] Go Version: go1.10
  3. INFO[0000] Go OS/Arch: darwin/amd64
  4. INFO[0000] operator-sdk Version: 0.0.5+git

3. Deploy your Operator with the Operator Lifecycle Manager (OLM)

OLM will manage creation of most if not all resources required to run your operator, using a bit of setup from other operator-sdk commands. Check out the OLM integration user guide for more information.

Create a Memcached CR

Modify deploy/crds/cache.example.com_v1alpha1_memcached_cr.yaml as shown and create a Memcached custom resource:

  1. $ cat deploy/crds/cache.example.com_v1alpha1_memcached_cr.yaml
  2. apiVersion: "cache.example.com/v1alpha1"
  3. kind: "Memcached"
  4. metadata:
  5.   name: "example-memcached"
  6. spec:
  7.   size: 3
  9. $ kubectl apply -f deploy/crds/cache.example.com_v1alpha1_memcached_cr.yaml

Ensure that the memcached-operator creates the deployment for the CR:

  1. $ kubectl get deployment
  2. NAME                     DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
  3. memcached-operator       1         1         1            1           2m
  4. example-memcached        3         3         3            3           1m

Check the pods to confirm 3 replicas were created:

  1. $ kubectl get pods
  2. NAME                                  READY     STATUS    RESTARTS   AGE
  3. example-memcached-6fd7c98d8-7dqdr     1/1       Running   0          1m
  4. example-memcached-6fd7c98d8-g5k7v     1/1       Running   0          1m
  5. example-memcached-6fd7c98d8-m7vn7     1/1       Running   0          1m
  6. memcached-operator-7cc7cfdf86-vvjqk   2/2       Running   0          2m

View the Ansible logs

In order to see the logs from a particular you can run:

  1. kubectl logs deployment/memcached-operator

The logs contain the information about the Ansible run and will make it much easier to debug issues within your Ansible tasks. Note that the logs will contain much more detailed information about the Ansible Operator’s internals and interface with Kubernetes as well.

Also, you can use the environment variable ANSIBLE_DEBUG_LOGS set as True to check the full Ansible result in the logs in order to be able to debug it.

Example

In the deploy/operator.yaml:

  1. ...
  2. - name: ANSIBLE_DEBUG_LOGS
  3.   value: "True"
  4. ...

Additional Ansible Debug

Occasionally while developing additional debug in the Operator logs is nice to have. Using the memcached operator as an example, we can simply add the "ansible.operator-sdk/verbosity" annotation to the Custom Resource with the desired verbosity.

  1. apiVersion: "cache.example.com/v1alpha1"
  2. kind: "Memcached"
  3. metadata:
  4.   name: "example-memcached"
  5.   annotations:
  6.     "ansible.operator-sdk/verbosity": "4"
  7. spec:
  8.   size: 4

Update the size

Change the spec.size field in the memcached CR from 3 to 4 and apply the change:

  1. $ cat deploy/crds/cache.example.com_v1alpha1_memcached_cr.yaml
  2. apiVersion: "cache.example.com/v1alpha1"
  3. kind: "Memcached"
  4. metadata:
  5.   name: "example-memcached"
  6. spec:
  7.   size: 4
  9. $ kubectl apply -f deploy/crds/cache.example.com_v1alpha1_memcached_cr.yaml

Confirm that the operator changes the deployment size:

  1. $ kubectl get deployment
  2. NAME                 DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
  3. example-memcached    4         4         4            4           5m

Cleanup

Clean up the resources:

  1. $ kubectl delete -f deploy/crds/cache.example.com_v1alpha1_memcached_cr.yaml
  2. $ kubectl delete -f deploy/operator.yaml
  3. $ kubectl delete -f deploy/role_binding.yaml
  4. $ kubectl delete -f deploy/role.yaml
  5. $ kubectl delete -f deploy/service_account.yaml
  6. $ kubectl delete -f deploy/crds/cache.example.com_memcacheds_crd.yaml

NOTE Additional CR/CRD’s can be added to the project by running, for example, the command :operator-sdk add api --api-version=cache.example.com/v1alpha1 --kind=AppService For more information, refer cli doc.

Last modified May 20, 2020: run: deprecate ‘—local’ in favor of ‘local’ subcommand (#3067) (ab710951)