Operator SDK Migration Guide from v0.0.x to v0.1.0

This document describes how to migrate an operator project built using Operator SDK v0.0.x to the project structure required by v0.1.0.

The recommended way to migrate your project is to initialize a new v0.1.0 project, then copy your code into the new project and modify as described below.

This guide goes over migrating the memcached-operator, an example project from the user guide, to illustrate migration steps. See the v0.0.7 memcached-operator and v0.1.0 memcached-operator project structures for pre- and post-migration examples, respectively.

Create a new v0.1.0 project

Rename your v0.0.x project and create a new v0.1.0 project in its place.

  1. # Ensure SDK version is v0.1.0
  2. $ operator-sdk --version
  3. operator-sdk version 0.1.0
  5. # Create new project
  6. $ cd $GOPATH/src/github.com/example-inc/
  7. $ mv memcached-operator old-memcached-operator
  8. $ operator-sdk new memcached-operator --skip-git-init
  9. $ ls
  10. memcached-operator old-memcached-operator
  12. # Copy over .git from old project
  13. $ cp -rf old-memcached-operator/.git memcached-operator/.git

Migrate custom types from pkg/apis

Scaffold api for custom types

Create the api for your custom resource (CR) in the new project with operator-sdk add api --api-version=<apiversion> --kind=<kind>

  1. $ cd memcached-operator
  2. $ operator-sdk add api --api-version=cache.example.com/v1alpha1 --kind=Memcached
  4. $ tree pkg/apis
  5. pkg/apis/
  6. ├── addtoscheme_cache_v1alpha1.go
  7. ├── apis.go
  8. └── cache
  9.     └── v1alpha1
  10.         ├── doc.go
  11.         ├── memcached_types.go
  12.         ├── register.go
  13.         └── zz_generated.deepcopy.go

Repeat the above command for as many custom types as you had defined in your old project. Each type will be defined in the file pkg/apis/<group>/<version>/<kind>_types.go.

Copy the contents of the type

Copy the Spec and Status contents of the pkg/apis/<group>/<version>/types.go file from the old project to the new project’s pkg/apis/<group>/<version>/<kind>_types.go file.

Note: Each <kind>_types.go file has an init() function. Be sure not to remove that since that registers the type with the Manager’s scheme.

  1. func init() {
  2.     SchemeBuilder.Register(&Memcached{}, &MemcachedList{})
  3. }

Migrate reconcile code

Add a controller to watch your CR

In a v0.0.x project you would define what resource to watch in cmd/<operator-name>/main.go

  1. sdk.Watch("cache.example.com/v1alpha1", "Memcached", "default", time.Duration(5)*time.Second)

For a v0.1.0 project you define a Controller to watch resources.

Add a controller to watch your CR type with operator-sdk add controller --api-version=<apiversion> --kind=<kind>.

  1. $ operator-sdk add controller --api-version=cache.example.com/v1alpha1 --kind=Memcached
  2. $ tree pkg/controller
  3. pkg/controller/
  4. ├── add_memcached.go
  5. ├── controller.go
  6. └── memcached
  7.     └── memcached_controller.go

Inspect the add() function in your pkg/controller/<kind>/<kind>_controller.go file:

  1. import (
  2.     cachev1alpha1 "github.com/example-inc/memcached-operator/pkg/apis/cache/v1alpha1"
  3.     ...
  4. )
  6. func add(mgr manager.Manager, r reconcile.Reconciler) error {
  7.     c, err := controller.New("memcached-controller", mgr, controller.Options{Reconciler: r})
  9.     // Watch for changes to the primary resource Memcached
  10.     err = c.Watch(&source.Kind{Type: &cachev1alpha1.Memcached{}}, &handler.EnqueueRequestForObject{})
  12.     // Watch for changes to the secondary resource Pods and enqueue reconcile requests for the owner Memcached
  13.     err = c.Watch(&source.Kind{Type: &corev1.Pod{}}, &handler.EnqueueRequestForOwner{
  14.         IsController: true,
  15.         OwnerType:    &cachev1alpha1.Memcached{},
  16.     })
  17. }

Remove the second Watch() or modify it to watch a secondary resource type that is owned by your CR.

Watching multiple resources lets you trigger the reconcile loop for multiple resources relevant to your application. See the watching and eventhandling doc and the Kubernetes controller conventions doc for more details.

Multiple custom resources

If your operator is watching more than 1 CR type then you can do one of the following depending on your application:

  • If the CR is owned by your primary CR then watch it as a secondary resource in the same controller to trigger the reconcile loop for the primary resource.

    1. // Watch for changes to the primary resource Memcached
    2. err = c.Watch(&source.Kind{Type: &cachev1alpha1.Memcached{}}, &handler.EnqueueRequestForObject{})
    4. // Watch for changes to the secondary resource AppService and enqueue reconcile requests for the owner Memcached
    5. err = c.Watch(&source.Kind{Type: &appv1alpha1.AppService{}}, &handler.EnqueueRequestForOwner{
    6.     IsController: true,
    7.     OwnerType:    &cachev1alpha1.Memcached{},
    8. })

  • Add a new controller to watch and reconcile the CR independently of the other CR.

    1. $ operator-sdk add controller --api-version=app.example.com/v1alpha1 --kind=AppService
    1. // Watch for changes to the primary resource AppService
    2. err = c.Watch(&source.Kind{Type: &appv1alpha1.AppService{}}, &handler.EnqueueRequestForObject{})

Copy and modify reconcile code from pkg/stub/handler.go

In a v0.1.0 project the reconcile code is defined in the Reconcile() method of a controller’s Reconciler. This is similar to the Handle() function in the older project. Note the difference in the arguments and return values:

  • Reconcile

    1. func (r *ReconcileMemcached) Reconcile(request reconcile.Request) (reconcile.Result, error)

  • Handle

    1. func (h *Handler) Handle(ctx context.Context, event sdk.Event) error

Instead of receiving an sdk.Event (with the object), the Reconcile() function receives a Request (Name/Namespace key) to lookup the object.

If the Reconcile() function returns an error, the controller will requeue and retry the Request. If no error is returned, then depending on the Result the controller will either not retry the Request, immediately retry, or retry after a specified duration.

Copy the code from the old project’s Handle() function over the existing code in your controller’s Reconcile() function. Be sure to keep the initial section in the Reconcile() code that looks up the object for the Request and checks to see if it’s deleted.

  1. import (
  2.     apierrors "k8s.io/apimachinery/pkg/api/errors"
  3.     cachev1alpha1 "github.com/example-inc/memcached-operator/pkg/apis/cache/v1alpha1"
  4.     ...
  5. )
  6. func (r *ReconcileMemcached) Reconcile(request reconcile.Request) (reconcile.Result, error) {
  7.     // Fetch the Memcached instance
  8.     instance := &cachev1alpha1.Memcached{}
  9.     err := r.client.Get(context.TODO()
  10.     request.NamespacedName, instance)
  11.     if err != nil {
  12.         if apierrors.IsNotFound(err) {
  13.             // Request object not found, could have been deleted after reconcile request.
  14.             // Owned objects are automatically garbage collected.
  15.             // Return and don't requeue
  16.             return reconcile.Result{}, nil
  17.         }
  18.         // Error reading the object - requeue the request.
  19.         return reconcile.Result{}, err
  20.     }
  22.     // Rest of your reconcile code goes here.
  23.     ...
  24. }

Update return values

Change the return values in your reconcile code:

  • Replace return err with return reconcile.Result{}, err
  • Replace return nil with return reconcile.Result{}, nil

Periodic reconcile

In order to periodically reconcile a CR in your controller you can set the RequeueAfter field for reconcile.Result. This will cause the controller to requeue the Request and trigger the reconcile after the desired duration. Note that the default value of 0 means no requeue.

  1. reconcilePeriod := 30 * time.Second
  2. reconcileResult := reconcile.Result{RequeueAfter: reconcilePeriod}
  3. ...
  5. // Update the status
  6. err := r.client.Update(context.TODO(), memcached)
  7. if err != nil {
  8.     log.Info(fmt.Sprintf("Failed to update memcached status: %v", err))
  9.     return reconcileResult, err
  10. }
  11. return reconcileResult, nil

Update client

Replace the calls to the SDK client(Create, Update, Delete, Get, List) with the reconciler’s client.

See the examples below and the controller-runtime client API doc for more details.

  1. // Create
  2. dep := &appsv1.Deployment{...}
  3. // v0.0.1
  4. err := sdk.Create(dep)
  5. // v0.1.0
  6. err := r.client.Create(context.TODO(), dep)
  8. // Update
  9. // v0.1.0
  10. err := sdk.Update(dep)
  11. // v0.1.0
  12. err := r.client.Update(context.TODO(), dep)
  14. // Delete
  15. err := sdk.Delete(dep)
  16. // v0.1.0
  17. err := r.client.Delete(context.TODO(), dep)
  19. // List
  20. podList := &corev1.PodList{}
  21. labelSelector := labels.SelectorFromSet(labelsForMemcached(memcached.Name))
  22. listOps := &metav1.ListOptions{LabelSelector: labelSelector}
  23. err := sdk.List(memcached.Namespace, podList, sdk.WithListOptions(listOps))
  24. // v0.1.0
  25. listOps := &client.ListOptions{Namespace: memcached.Namespace, LabelSelector: labelSelector}
  26. err := r.client.List(context.TODO(), listOps, podList)
  28. // Get
  29. dep := &appsv1.Deployment{APIVersion: "apps/v1", Kind: "Deployment", Name: name, Namespace: namespace}
  30. err := sdk.Get(dep)
  31. // v0.1.0
  32. dep := &appsv1.Deployment{}
  33. err = r.client.Get(context.TODO(), types.NamespacedName{Name: name, Namespace: namespace}, dep)
  34. // v0.1.0 with unstructured
  35. dep := &unstructured.Unstructured{}
  36. dep.SetGroupVersionKind(schema.GroupVersionKind{Group:"apps", Version: "v1", Kind:"Deployment"})
  37. err = r.client.Get(context.TODO(), types.NamespacedName{Name: name, Namespace: namespace}, dep)

Lastly copy and initialize any other fields that you may have had in your Handler struct into the Reconcile<Kind> struct:

  1. // newReconciler returns a new reconcile.Reconciler
  2. func newReconciler(mgr manager.Manager) reconcile.Reconciler {
  3.     return &ReconcileMemcached{client: mgr.GetClient(), scheme: mgr.GetScheme(), foo: "bar"}
  4. }
  6. // ReconcileMemcached reconciles a Memcached object
  7. type ReconcileMemcached struct {
  8.     client client.Client
  9.     scheme *runtime.Scheme
  10.     // Other fields
  11.     foo string
  12. }

Copy changes from main.go

The main function for a v0.1.0 operator in cmd/manager/main.go sets up the Manager which registers the custom resources and starts all the controllers.

There is no need to migrate the SDK functions sdk.Watch(),sdk.Handle(), and sdk.Run() from the old main.go since that logic is now defined in a controller.

However if there are any operator specific flags or settings defined in the old main file copy those over.

If you have any 3rd party resource types registered with the SDK’s scheme, then register those with the Manager’s scheme in the new project. See how to register 3rd party resources.

operator-sdk now expects cmd/manager/main.go to be present in Go operator projects. Go project-specific commands, ex. add [api, controller], will error if main.go is not found in its expected path.

Copy user defined files

If there are any user defined pkgs, scripts, and docs in the older project, copy these files into the new project.

Copy changes to deployment manifests

For any updates made to the following manifests in the old project, copy over the changes to their corresponding files in the new project. Be careful not to directly overwrite the files but inspect and make any changes necessary.

  • tmp/build/Dockerfile to build/Dockerfile
    • There is no tmp directory in the new project layout
  • RBAC rules updates from deploy/rbac.yaml to deploy/role.yaml and deploy/role_binding.yaml
  • deploy/cr.yaml to deploy/crds/<full group>_<version>_<kind>_cr.yaml
  • deploy/crd.yaml to deploy/crds/<group>_<version>_<kind>_crd.yaml

Copy user defined dependencies

For any user defined dependencies added to the old project’s Gopkg.toml, copy and append them to the new project’s Gopkg.toml. Run dep ensure to update the vendor in the new project.

Confirmation

At this point you should be able to build and run your operator to verify that it works. See the user-guide on how to build and run your operator.

Last modified January 1, 0001