Common recommendations and suggestions

Common recommendations and suggestions to built solutions with Operator SDK

Overview

Any recommendations or best practices suggested by the Kubernetes community, such as how to develop Operator pattern solutions or how to use controller-runtime are good recommendations for those who are looking to build operator projects with operator-sdk. Also, see Operator Best Practices. However, here are some common recommendations.

Common Recommendations

Develop idempotent reconciliation solutions

When developing operators, it is essential for the controller’s reconciliation loop to be idempotent. By following the Operator pattern you will create Controllers which provide a reconcile function responsible for synchronizing resources until the desired state is reached on the cluster. Breaking this recommendation goes against the design principles of controller-runtime and may lead to unforeseen consequences such as resources becoming stuck and requiring manual intervention.

Understanding Kubernetes APIs

Building your own operator commonly involves extending the Kubernetes API itself. It is helpful to understand exactly how Custom Resource Definitions interact with the Kubernetes API. Also, the Kubebuilder documentation on Groups and Versions and Kinds may be helpful to better understand these concepts as they relate to operators.

Avoid a design solution where more than one Kind is reconciled by the same controller

Having many Kinds (such as CRDs) which are all managed by the same controller usually goes against the design proposed by controller-runtime. Furthermore this might hurt concepts such as encapsulation, the Single Responsibility Principle, and Cohesion. Damaging these concepts may cause unexpected side effects, and increase the difficulty of extending, reusing, or maintaining the operator.

Other common suggestions

• Provide the images and tags used by the operator solution via environment variables in the config/manager/manager.yaml:

...
spec:
  ...
    spec:
      ...
      containers:
      - command:
        - /manager
        ...
        env:
        - name: MY_IMAGE
          value: "quay.io/example.com/image:0.0.1"

• Manage your solutions using Status Conditionals
• Use finalizers when/if required
• Cover the project with tests/CI to ensure its quality:
  • For any language-based operator, you can use Scorecard to implement functional tests
  • For Go-based operators, you can also use envtest to cover the controllers. For further information see Testing with EnvTest
  • For Ansible-based operators, you can also use Molecule, an Ansible testing framework. For further information see Testing with Molecule
  • For Helm-based operators, you can also use Chart tests
• Ensure that you checked the Can I customize the projects initialized with operator-sdk? and understand the Project Layout before starting to do your customizations as please you on top.
• Optimize manager resource values in config/manager/manager.yaml according to project requirements. It is recommended to define resources limits in order to follow good practices and for security reasons. More info: Managing Resources for Containers and Docker Security Cheat Sheet.
• Look for TODO(user) in the source code generated by the CLI to ensure that you follow all suggested customizations.
• If you wish to integrate your project with OLM, you can also check its Best Practices section.

Last modified November 11, 2021: docs [bestpractices/common suggestions]: small typo nit (#5352) (e60d4ccc)