Generating CRD

Kubebuilder provides a tool named controller-gen to generate manifests for CustomResourceDefinitions. The tool resides in the controller-tools repository which is vendored in every project. If you examine the Makefile in the project, you will see a target named manifests as shown below. manifests target is also listed as prerequisite for other targets like run, tests, deploy etc to ensure CRD manifests are regenerated when needed.

  1. # Generate manifests e.g. CRD, RBAC etc.
  2. manifests:
  3.     go run vendor/sigs.k8s.io/controller-tools/cmd/controller-gen/main.go all

When you run make manifests, you should see an output like below. CRDs are generated under configs/crds directory.

  1. $ make manifests
  2. go run vendor/sigs.k8s.io/controller-tools/cmd/controller-gen/main.go all
  3. CRD manifests generated under '<curr_dir>/config/crds'
  4. RBAC manifests generated under '<curr_dir>/config/rbac'

Though controller-gen generates manifests for RBAC as well, but this section describes the generation of CRD manifest.

controller-gen reads kubebuilder annotations of the form // +kubebuilder:something… defined as Go comments in the <your-api-kind>_types.go file under pkg/apis/… to produce the CRD manifests. Sections below describe various supported annotations.

Validation

CRDs support validation by definining (OpenAPI v3 schema) in the validation section. To learn more about the validation feature, refer to the original docs here. One can specify validation for a field by annotating the field with kubebuilder annotation which is of the form// +kubebuilder:validation:<key=value>. If you want to specify multiple validations for a field, you can add multiple such annotation as demonstrated in the example below.

Currently, supporting keys are Maximum, Minimum, MaxLength, MinLength, MaxItems, MinItems, UniqueItems, Enum, Pattern, ExclusiveMaximum, ExclusiveMinimum, MultipleOf, Format. The // +kubebuilder:validation:Pattern=.+:.+ annotation specifies the Pattern validation requiring that the Image field match the regular expression .+:.+

Example:

  1. type ToySpec struct {
  3.     // +kubebuilder:validation:Maximum=100
  4.     // +kubebuilder:validation:Minimum=1
  5.     // +kubebuilder:validation:ExclusiveMinimum=true
  6.     Power  float32 `json:"power,omitempty"`
  8.     Bricks int32   `json:"bricks,omitempty"`
  9.     // +kubebuilder:validation:MaxLength=15
  10.     // +kubebuilder:validation:MinLength=1
  11.     Name string `json:"name,omitempty"`
  13.     // +kubebuilder:validation:MaxItems=500
  14.     // +kubebuilder:validation:MinItems=1
  15.     // +kubebuilder:validation:UniqueItems=false
  16.     Knights []string `json:"knights,omitempty"`
  18.     // +kubebuilder:validation:Enum=Lion,Wolf,Dragon
  19.     Alias string `json:"alias,omitempty"`
  21.     // +kubebuilder:validation:Enum=1,2,3
  22.     Rank    int    `json:"rank"`
  23. }

Additional printer columns

Starting with Kubernetes 1.11, kubectl uses server-side printing. The serverdecides which columns are shown by the kubectl get command. You can customize these columns using a CustomResourceDefinition.To add an additional column, add a comment with the following annotation formatjust above the struct definition of the Kind.

Format: // +kubebuilder:printcolumn:name="Name",type="type",JSONPath="json-path",description="desc",priority="priority",format="format"

Note that description, priority and format are optional. Refer to theadditonal printer columns docsto learn more about the values of name, type, JsonPath, description, priority and format.

The following example adds the Spec, Replicas, and Age columns.

  1. // +kubebuilder:printcolumn:name="Spec",type="integer",JSONPath=".spec.cronSpec",description="status of the kind"
  2. // +kubebuilder:printcolumn:name="Replicas",type="integer",JSONPath=".spec.Replicas",description="The number of jobs launched by the CronJob"
  3. // +kubebuilder:printcolumn:name="Age",type="date",JSONPath=".metadata.creationTimestamp"
  4. type CronTab struct {
  5.     metav1.TypeMeta   `json:",inline"`
  6.     metav1.ObjectMeta `json:"metadata,omitempty"`
  8.     Spec   CronTabSpec   `json:"spec,omitempty"`
  9.     Status CronTabStatus `json:"status,omitempty"`
  10. }

Subresource

Custom resources support /status and /scale subresources as of kubernetes1.13 release. You can learn more about the subresources here.

1. Status

To enable /status subresource, annotate the kind with // +kubebuilder:subresource:status format.

2. Scale

To enable /scale subresource, annotate the kind with // +kubebuilder:subresource:scale:specpath=<jsonpath>,statuspath=<jsonpath>,selectorpath=<jsonpath> format.

Scale subresource annotation contains three fields: specpath, statuspath and selectorpath.

  • specpath refers to specReplicasPath attribute of Scale object, and value jsonpath defines the JSONPath inside of a custom resource that corresponds to Scale.Spec.Replicas. This is a required field.
  • statuspath refers to statusReplicasPath attribute of Scale object. and the jsonpath value of it defines the JSONPath inside of a custom resource that corresponds to Scale.Status.Replicas. This is a required field.
  • selectorpath refers to labelSelectorPath attribute of Scale object, and the value jsonpath defines the JSONPath inside of a custom resource that corresponds to Scale.Status.Selector. This is an optional field.Example:
  1. type ToySpec struct {
  2.     Replicas *int32 `json:"replicas"` // Add this field in Toy Spec, so the jsonpath to this field is `.spec.replicas`
  3. }
  5. // ToyStatus defines the observed state of Toy
  6. type ToyStatus struct {
  7.     Replicas int32 `json:"replicas"` // Add this field in Toy Status, so the jsonpath to this field is `.status.replicas`
  8. }
  11. // Toy is the Schema for the toys API
  12. // +k8s:openapi-gen=true
  13. // +kubebuilder:subresource:status
  14. // +kubebuilder:subresource:scale:specpath=.spec.replicas,statuspath=.status.replicas
  15. type Toy struct {
  16.     metav1.TypeMeta   `json:",inline"`
  17.     metav1.ObjectMeta `json:"metadata,omitempty"`
  19.     Spec   ToySpec   `json:"spec,omitempty"`
  20.     Status ToyStatus `json:"status,omitempty"`
  21. }

In order to enable scale subresource in type definition file, you have to apply the scale subresource right before the kind struct definition, with correct jsonpath values according to the spec and status. And then make sure the jsonpaths are already defined in the Spec and Status struct. Finally, update the <kind>_types_test.go files according to the types Spec and Status changes.

In the above example for the type Toy, we added // +kubebuilder:subresource:scale:specpath=.spec.replicas,statuspath=.status.replicas comment before Toy struct definition. .spec.replicas refers to the josnpath of Spec struct field (ToySpec.Replicas). And jsonpath .status.healthyReplicas refers to Status struct field (ToyStatus.Replicas).