Kubernetes CRD Extensions

Explains extension points for deep CRD integrations within Spinnaker.

Spinnaker Extension Points for Custom Resource Definitions

At Google, we’ve built extension points for deep CRD integrations within Spinnaker.

This work has allowed us to support the following features within Spinnaker:

  • Custom models for representing CRDs as spinnakerKinds
  • Deploying CRDs with custom Spinnaker artifact types
  • Custom Kubernetes API versions
  • Custom Spinnaker naming strategies
  • Per-account, custom Spinnaker UIs that can run alongside the existing Kubernetes UIs

This guide is for developers who want to duplicate this functionality for their CRDs. It also exists as an explanation of certain code paths within Spinnaker which include hooks with no current corresponding open-source implementations.

Developers who want to implement these features will have to build their own layered version of Clouddriver - see Adam Jorden’s blog post - and should be familiar with the Kubernetes provider and writing code for Clouddriver.

Custom Handlers

The central extension point is the KubernetesHandler class. A subclass of KubernetesHandler - e.g., KubernetesReplicaSetHandler - defines the relationship between Spinnaker and your Kubernetes kind.

For example, if you wanted to build a Spinnaker integration for your CRD of kind MyCRDKind, you would start with the following handler:

  1. @Component
  2. public class MyCRDHandler extends KubernetesHandler {
  4.   public MyCRDHandler() {
  5.     // Hook point for registering a custom `ArtifactReplacer`
  6.     // for your CRD. During a deploy operation,
  7.     // if an artifact of the type specified in the replacer is present,
  8.     // the artifact will be inserted into the manifest using the
  9.     // strategy described in the replacer.
  10.   }
  12.   @Override
  13.   public KubernetesKind kind() {
  14.     return KubernetesKind.from("MyCRDKind");
  15.   }
  17.   @Override
  18.   public boolean versioned() {
  19.     // If the CRD resource should be versioned - i.e., assigned a sequence
  20.     // v001, v002, etc.
  21.     return false;
  22.   }
  24.   @Override
  25.   public SpinnakerKind spinnakerKind() {
  26.     // The Spinnaker kind that the resource will be represented as in Spinnaker's API and UI.
  27.     return SpinnakerKind.SERVER_GROUPS;
  28.   }
  30.   @Override
  31.   public Status status(KubernetesManifest manifest) {
  32.     // Includes logic for determining whether your CRD manifest is stable.
  33.     // A deploy manifest operation, for example, will block until this
  34.     // method returns a stable status.
  35.   }
  37.   @Override
  38.   public KubernetesV2CachingAgentFactory cachingAgentFactory() {
  39.     return KubernetesCoreCachingAgent::new;
  40.   }
  41. }

Custom Spinnaker Resource Models

You may want to change how their CRD is represented in Spinnaker’s API. By default, a CRD of spinnakerKind serverGroups will be represented with the model class KubernetesV2ServerGroup .

You may want to override this representation, for example, if you want to define how your server group’s region is resolved.

To override the default model class, MyCRDHandler should implement ServerGroupHandler (or ServerGroupManagerHandler if your resource is of spinnakerKind serverGroupManagers). You will be responsible for translating raw Spinnaker cache data into a subclass of KubernetesServerGroup.

Custom Kubernetes API Versions

If you want Spinnaker to support custom Kubernetes API versions, subclass KubernetesApiVersion.

For example,

  1. public class MyApiVersion extends KubernetesApiVersion {
  2.   public static MY_BETA_API_VERSION = new MyApiVersion("myApiVersion/v1beta");
  4.   public MyApiVersion(String name) {
  5.     // Base class maintains state.
  6.     super(name);
  7.   }
  8. }

Custom Spinnaker Naming Strategies

To use a custom naming strategy for your CRD, implement NamingStrategy . For example,

  1. @Component
  2. public class MyManifestNamer implements NamingStrategy<KubernetesManifest> {
  3.   @Override
  4.   public String getName() {
  5.     return "myManifestNamingStrategy";
  6.   }
  8.   @Override
  9.   public void applyMoniker(KubernetesManifest manifest, Moniker moniker) {
  10.     // Strategy for applying a Spinnaker `Moniker` to a Kubernetes
  11.     // manifest prior to deployment.
  12.   }
  14.   @Override
  15.   public Moniker deriveMoniker(KubernetesManifest manifest) {
  16.     // Strategy for deriving a Spinnaker `Moniker` from a Kubernetes
  17.     // manifest.
  18.   }
  19. }

This naming strategy can be referenced in a Kubernetes account config. For example:

  1. kubernetes:
  2.   accounts:
  3.     - name: my-kubernetes-account
  4.       namingStrategy: myManifestNamingStrategy

Be careful - this naming strategy will be applied to all manifests manipulated by this account.

Last modified June 24, 2021: Redesign Progress (#83) (8231bcf)