Declarative Management of Kubernetes Objects Using Kustomize

Kustomize is a standalone tool to customize Kubernetes objects through a kustomization file.

Since 1.14, Kubectl also supports the management of Kubernetes objects using a kustomization file. To view Resources found in a directory containing a kustomization file, run the following command:

  1. kubectl kustomize <kustomization_directory>

To apply those Resources, run kubectl apply with --kustomize or -k flag:

  1. kubectl apply -k <kustomization_directory>

Before you begin

Install kubectl.

You need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using Minikube, or you can use one of these Kubernetes playgrounds:

To check the version, enter kubectl version.

Overview of Kustomize

Kustomize is a tool for customizing Kubernetes configurations. It has the following features to manage application configuration files:

  • generating resources from other sources
  • setting cross-cutting fields for resources
  • composing and customizing collections of resources

Generating Resources

ConfigMap and Secret hold config or sensitive data that are used by other Kubernetes objects, such as Pods. The source of truth of ConfigMap or Secret are usually from somewhere else, such as a .properties file or a ssh key file. Kustomize has secretGenerator and configMapGenerator, which generate Secret and ConfigMap from files or literals.

configMapGenerator

To generate a ConfigMap from a file, add an entry to files list in configMapGenerator. Here is an example of generating a ConfigMap with a data item from a file content.

  1. # Create a application.properties file
  2. cat <<EOF >application.properties
  3. FOO=Bar
  4. EOF
  6. cat <<EOF >./kustomization.yaml
  7. configMapGenerator:
  8. - name: example-configmap-1
  9.   files:
  10.   - application.properties
  11. EOF

The generated ConfigMap can be checked by the following command:

  1. kubectl kustomize ./

The generated ConfigMap is:

  1. apiVersion: v1
  2. data:
  3.   application.properties: |
  4.     FOO=Bar
  5. kind: ConfigMap
  6. metadata:
  7.   name: example-configmap-1-8mbdf7882g

ConfigMap can also be generated from literal key-value pairs. To generate a ConfigMap from a literal key-value pair, add an entry to literals list in configMapGenerator. Here is an example of generating a ConfigMap with a data item from a key-value pair.

  1. cat <<EOF >./kustomization.yaml
  2. configMapGenerator:
  3. - name: example-configmap-2
  4.   literals:
  5.   - FOO=Bar
  6. EOF

The generated ConfigMap can be checked by the following command:

  1. kubectl kustomize ./

The generated ConfigMap is

  1. apiVersion: v1
  2. data:
  3.   FOO: Bar
  4. kind: ConfigMap
  5. metadata:
  6.   name: example-configmap-2-g2hdhfc6tk

secretGenerator

You can generate Secrets from files or literal key-value pairs. To generate a Secret from a file, add an entry to files list in secretGenerator. Here is an example of generating a Secret with a data item from a file.

  1. # Create a password.txt file
  2. cat <<EOF >./password.txt
  3. username=admin
  4. password=secret
  5. EOF
  7. cat <<EOF >./kustomization.yaml
  8. secretGenerator:
  9. - name: example-secret-1
  10.   files:
  11.   - password.txt
  12. EOF

The generated Secret is as follows:

  1. apiVersion: v1
  2. data:
  3.   password.txt: dXNlcm5hbWU9YWRtaW4KcGFzc3dvcmQ9c2VjcmV0Cg==
  4. kind: Secret
  5. metadata:
  6.   name: example-secret-1-t2kt65hgtb
  7. type: Opaque

To generate a Secret from a literal key-value pair, add an entry to literals list in secretGenerator. Here is an example of generating a Secret with a data item from a key-value pair.

  1. cat <<EOF >./kustomization.yaml
  2. secretGenerator:
  3. - name: example-secret-2
  4.   literals:
  5.   - username=admin
  6.   - password=secret
  7. EOF

The generated Secret is as follows:

  1. apiVersion: v1
  2. data:
  3.   password: c2VjcmV0
  4.   username: YWRtaW4=
  5. kind: Secret
  6. metadata:
  7.   name: example-secret-2-t52t6g96d8
  8. type: Opaque

generatorOptions

The generated ConfigMaps and Secrets have a suffix appended by hashing the contents. This ensures that a new ConfigMap or Secret is generated when the content is changed. To disable the behavior of appending a suffix, one can use generatorOptions. Besides that, it is also possible to specify cross-cutting options for generated ConfigMaps and Secrets.

  1. cat <<EOF >./kustomization.yaml
  2. configMapGenerator:
  3. - name: example-configmap-3
  4.   literals:
  5.   - FOO=Bar
  6. generatorOptions:
  7.   disableNameSuffixHash: true
  8.   labels:
  9.     type: generated
  10.   annotations:
  11.     note: generated
  12. EOF

Runkubectl kustomize ./ to view the generated ConfigMap:

  1. apiVersion: v1
  2. data:
  3.   FOO: Bar
  4. kind: ConfigMap
  5. metadata:
  6.   annotations:
  7.     note: generated
  8.   labels:
  9.     type: generated
  10.   name: example-configmap-3

Setting cross-cutting fields

It is quite common to set cross-cutting fields for all Kubernetes resources in a project. Some use cases for setting cross-cutting fields:

  • setting the same namespace for all Resource
  • adding the same name prefix or suffix
  • adding the same set of labels
  • adding the same set of annotations

Here is an example:

  1. # Create a deployment.yaml
  2. cat <<EOF >./deployment.yaml
  3. apiVersion: apps/v1
  4. kind: Deployment
  5. metadata:
  6.   name: nginx-deployment
  7.   labels:
  8.     app: nginx
  9. spec:
  10.   selector:
  11.     matchLabels:
  12.       app: nginx
  13.   template:
  14.     metadata:
  15.       labels:
  16.         app: nginx
  17.     spec:
  18.       containers:
  19.       - name: nginx
  20.         image: nginx
  21. EOF
  23. cat <<EOF >./kustomization.yaml
  24. namespace: my-namespace
  25. namePrefix: dev-
  26. nameSuffix: "-001"
  27. commonLabels:
  28.   app: bingo
  29. commonAnnotations:
  30.   oncallPager: 800-555-1212
  31. resources:
  32. - deployment.yaml
  33. EOF

Run kubectl kustomize ./ to view those fields are all set in the Deployment Resource:

  1. apiVersion: apps/v1
  2. kind: Deployment
  3. metadata:
  4.   annotations:
  5.     oncallPager: 800-555-1212
  6.   labels:
  7.     app: bingo
  8.   name: dev-nginx-deployment-001
  9.   namespace: my-namespace
  10. spec:
  11.   selector:
  12.     matchLabels:
  13.       app: bingo
  14.   template:
  15.     metadata:
  16.       annotations:
  17.         oncallPager: 800-555-1212
  18.       labels:
  19.         app: bingo
  20.     spec:
  21.       containers:
  22.       - image: nginx
  23.         name: nginx

Composing and Customizing Resources

It is common to compose a set of Resources in a project and manage them inside the same file or directory. Kustomize offers composing Resources from different files and applying patches or other customization to them.

Composing

Kustomize supports composition of different resources. The resources field, in the kustomization.yaml file, defines the list of resources to include in a configuration. Set the path to a resource’s configuration file in the resources list. Here is an example for an nginx application with a Deployment and a Service.

  1. # Create a deployment.yaml file
  2. cat <<EOF > deployment.yaml
  3. apiVersion: apps/v1
  4. kind: Deployment
  5. metadata:
  6.   name: my-nginx
  7. spec:
  8.   selector:
  9.     matchLabels:
  10.       run: my-nginx
  11.   replicas: 2
  12.   template:
  13.     metadata:
  14.       labels:
  15.         run: my-nginx
  16.     spec:
  17.       containers:
  18.       - name: my-nginx
  19.         image: nginx
  20.         ports:
  21.         - containerPort: 80
  22. EOF
  24. # Create a service.yaml file
  25. cat <<EOF > service.yaml
  26. apiVersion: v1
  27. kind: Service
  28. metadata:
  29.   name: my-nginx
  30.   labels:
  31.     run: my-nginx
  32. spec:
  33.   ports:
  34.   - port: 80
  35.     protocol: TCP
  36.   selector:
  37.     run: my-nginx
  38. EOF
  40. # Create a kustomization.yaml composing them
  41. cat <<EOF >./kustomization.yaml
  42. resources:
  43. - deployment.yaml
  44. - service.yaml
  45. EOF

The Resources from kubectl kustomize ./ contains both the Deployment and the Service objects.

Customizing

On top of Resources, one can apply different customizations by applying patches. Kustomize supports different patching mechanisms through patchesStrategicMerge and patchesJson6902. patchesStrategicMerge is a list of file paths. Each file should be resolved to a strategic merge patch. The names inside the patches must match Resource names that are already loaded. Small patches that do one thing are recommended. For example, create one patch for increasing the deployment replica number and another patch for setting the memory limit.

  1. # Create a deployment.yaml file
  2. cat <<EOF > deployment.yaml
  3. apiVersion: apps/v1
  4. kind: Deployment
  5. metadata:
  6.   name: my-nginx
  7. spec:
  8.   selector:
  9.     matchLabels:
  10.       run: my-nginx
  11.   replicas: 2
  12.   template:
  13.     metadata:
  14.       labels:
  15.         run: my-nginx
  16.     spec:
  17.       containers:
  18.       - name: my-nginx
  19.         image: nginx
  20.         ports:
  21.         - containerPort: 80
  22. EOF
  24. # Create a patch increase_replicas.yaml
  25. cat <<EOF > increase_replicas.yaml
  26. apiVersion: apps/v1
  27. kind: Deployment
  28. metadata:
  29.   name: my-nginx
  30. spec:
  31.   replicas: 3
  32. EOF
  34. # Create another patch set_memory.yaml
  35. cat <<EOF > set_memory.yaml
  36. apiVersion: apps/v1
  37. kind: Deployment
  38. metadata:
  39.   name: my-nginx
  40. spec:
  41.   template:
  42.     spec:
  43.       containers:
  44.       - name: my-nginx
  45.         resources:
  46.         limits:
  47.           memory: 512Mi
  48. EOF
  50. cat <<EOF >./kustomization.yaml
  51. resources:
  52. - deployment.yaml
  53. patchesStrategicMerge:
  54. - increase_replicas.yaml
  55. - set_memory.yaml
  56. EOF

Run kubectl kustomize ./ to view the Deployment:

  1. apiVersion: apps/v1
  2. kind: Deployment
  3. metadata:
  4.   name: my-nginx
  5. spec:
  6.   replicas: 3
  7.   selector:
  8.     matchLabels:
  9.       run: my-nginx
  10.   template:
  11.     metadata:
  12.       labels:
  13.         run: my-nginx
  14.     spec:
  15.       containers:
  16.       - image: nginx
  17.         limits:
  18.           memory: 512Mi
  19.         name: my-nginx
  20.         ports:
  21.         - containerPort: 80

Not all Resources or fields support strategic merge patches. To support modifying arbitrary fields in arbitrary Resources, Kustomize offers applying JSON patch through patchesJson6902. To find the correct Resource for a Json patch, the group, version, kind and name of that Resource need to be specified in kustomization.yaml. For example, increasing the replica number of a Deployment object can also be done through patchesJson6902.

  1. # Create a deployment.yaml file
  2. cat <<EOF > deployment.yaml
  3. apiVersion: apps/v1
  4. kind: Deployment
  5. metadata:
  6.   name: my-nginx
  7. spec:
  8.   selector:
  9.     matchLabels:
  10.       run: my-nginx
  11.   replicas: 2
  12.   template:
  13.     metadata:
  14.       labels:
  15.         run: my-nginx
  16.     spec:
  17.       containers:
  18.       - name: my-nginx
  19.         image: nginx
  20.         ports:
  21.         - containerPort: 80
  22. EOF
  24. # Create a json patch
  25. cat <<EOF > patch.yaml
  26. - op: replace
  27.   path: /spec/replicas
  28.   value: 3
  29. EOF
  31. # Create a kustomization.yaml
  32. cat <<EOF >./kustomization.yaml
  33. resources:
  34. - deployment.yaml
  36. patchesJson6902:
  37. - target:
  38.     group: apps
  39.     version: v1
  40.     kind: Deployment
  41.     name: my-nginx
  42.   path: patch.yaml
  43. EOF

Run kubectl kustomize ./ to see the replicas field is updated:

  1. apiVersion: apps/v1
  2. kind: Deployment
  3. metadata:
  4.   name: my-nginx
  5. spec:
  6.   replicas: 3
  7.   selector:
  8.     matchLabels:
  9.       run: my-nginx
  10.   template:
  11.     metadata:
  12.       labels:
  13.         run: my-nginx
  14.     spec:
  15.       containers:
  16.       - image: nginx
  17.         name: my-nginx
  18.         ports:
  19.         - containerPort: 80

In addition to patches, Kustomize also offers customizing container images or injecting field values from other objects into containers without creating patches. For example, you can change the image used inside containers by specifying the new image in images field in kustomization.yaml.

  1. cat <<EOF > deployment.yaml
  2. apiVersion: apps/v1
  3. kind: Deployment
  4. metadata:
  5.   name: my-nginx
  6. spec:
  7.   selector:
  8.     matchLabels:
  9.       run: my-nginx
  10.   replicas: 2
  11.   template:
  12.     metadata:
  13.       labels:
  14.         run: my-nginx
  15.     spec:
  16.       containers:
  17.       - name: my-nginx
  18.         image: nginx
  19.         ports:
  20.         - containerPort: 80
  21. EOF
  23. cat <<EOF >./kustomization.yaml
  24. resources:
  25. - deployment.yaml
  26. images:
  27. - name: nginx
  28.   newName: my.image.registry/nginx
  29.   newTag: 1.4.0
  30. EOF

Run kubectl kustomize ./ to see that the image being used is updated:

  1. apiVersion: apps/v1
  2. kind: Deployment
  3. metadata:
  4.   name: my-nginx
  5. spec:
  6.   replicas: 2
  7.   selector:
  8.     matchLabels:
  9.       run: my-nginx
  10.   template:
  11.     metadata:
  12.       labels:
  13.         run: my-nginx
  14.     spec:
  15.       containers:
  16.       - image: my.image.registry/nginx:1.4.0
  17.         name: my-nginx
  18.         ports:
  19.         - containerPort: 80

Sometimes, the application running in a Pod may need to use configuration values from other objects. For example, a Pod from a Deployment object need to read the corresponding Service name from Env or as a command argument. Since the Service name may change as namePrefix or nameSuffix is added in the kustomization.yaml file. It is not recommended to hard code the Service name in the command argument. For this usage, Kustomize can inject the Service name into containers through vars.

  1. # Create a deployment.yaml file
  2. cat <<EOF > deployment.yaml
  3. apiVersion: apps/v1
  4. kind: Deployment
  5. metadata:
  6.   name: my-nginx
  7. spec:
  8.   selector:
  9.     matchLabels:
  10.       run: my-nginx
  11.   replicas: 2
  12.   template:
  13.     metadata:
  14.       labels:
  15.         run: my-nginx
  16.     spec:
  17.       containers:
  18.       - name: my-nginx
  19.         image: nginx
  20.         command: ["start", "--host", "\$(MY_SERVICE_NAME)"]
  21. EOF
  23. # Create a service.yaml file
  24. cat <<EOF > service.yaml
  25. apiVersion: v1
  26. kind: Service
  27. metadata:
  28.   name: my-nginx
  29.   labels:
  30.     run: my-nginx
  31. spec:
  32.   ports:
  33.   - port: 80
  34.     protocol: TCP
  35.   selector:
  36.     run: my-nginx
  37. EOF
  39. cat <<EOF >./kustomization.yaml
  40. namePrefix: dev-
  41. nameSuffix: "-001"
  43. resources:
  44. - deployment.yaml
  45. - service.yaml
  47. vars:
  48. - name: MY_SERVICE_NAME
  49.   objref:
  50.     kind: Service
  51.     name: my-nginx
  52.     apiVersion: v1
  53. EOF

Run kubectl kustomize ./ to see that the Service name injected into containers is dev-my-nginx-001:

  1. apiVersion: apps/v1
  2. kind: Deployment
  3. metadata:
  4.   name: dev-my-nginx-001
  5. spec:
  6.   replicas: 2
  7.   selector:
  8.     matchLabels:
  9.       run: my-nginx
  10.   template:
  11.     metadata:
  12.       labels:
  13.         run: my-nginx
  14.     spec:
  15.       containers:
  16.       - command:
  17.         - start
  18.         - --host
  19.         - dev-my-nginx-001
  20.         image: nginx
  21.         name: my-nginx

Bases and Overlays

Kustomize has the concepts of bases and overlays. A base is a directory with a kustomization.yaml, which contains a set of resources and associated customization. A base could be either a local directory or a directory from a remote repo, as long as a kustomization.yaml is present inside. An overlay is a directory with a kustomization.yaml that refers to other kustomization directories as its bases. A base has no knowledge of an overlay and can be used in multiple overlays. An overlay may have multiple bases and it composes all resources from bases and may also have customization on top of them.

Here is an example of a base:

  1. # Create a directory to hold the base
  2. mkdir base
  3. # Create a base/deployment.yaml
  4. cat <<EOF > base/deployment.yaml
  5. apiVersion: apps/v1
  6. kind: Deployment
  7. metadata:
  8.   name: my-nginx
  9. spec:
  10.   selector:
  11.     matchLabels:
  12.       run: my-nginx
  13.   replicas: 2
  14.   template:
  15.     metadata:
  16.       labels:
  17.         run: my-nginx
  18.     spec:
  19.       containers:
  20.       - name: my-nginx
  21.         image: nginx
  22. EOF
  24. # Create a base/service.yaml file
  25. cat <<EOF > base/service.yaml
  26. apiVersion: v1
  27. kind: Service
  28. metadata:
  29.   name: my-nginx
  30.   labels:
  31.     run: my-nginx
  32. spec:
  33.   ports:
  34.   - port: 80
  35.     protocol: TCP
  36.   selector:
  37.     run: my-nginx
  38. EOF
  39. # Create a base/kustomization.yaml
  40. cat <<EOF > base/kustomization.yaml
  41. resources:
  42. - deployment.yaml
  43. - service.yaml
  44. EOF

This base can be used in multiple overlays. You can add different namePrefix or other cross-cutting fields in different overlays. Here are two overlays using the same base.

  1. mkdir dev
  2. cat <<EOF > dev/kustomization.yaml
  3. bases:
  4. - ../base
  5. namePrefix: dev-
  6. EOF
  8. mkdir prod
  9. cat <<EOF > prod/kustomization.yaml
  10. bases:
  11. - ../base
  12. namePrefix: prod-
  13. EOF

How to apply/view/delete objects using Kustomize

Use --kustomize or -k in kubectl commands to recognize Resources managed by kustomization.yaml. Note that -k should point to a kustomization directory, such as

  1. kubectl apply -k <kustomization directory>/

Given the following kustomization.yaml,

  1. # Create a deployment.yaml file
  2. cat <<EOF > deployment.yaml
  3. apiVersion: apps/v1
  4. kind: Deployment
  5. metadata:
  6.   name: my-nginx
  7. spec:
  8.   selector:
  9.     matchLabels:
  10.       run: my-nginx
  11.   replicas: 2
  12.   template:
  13.     metadata:
  14.       labels:
  15.         run: my-nginx
  16.     spec:
  17.       containers:
  18.       - name: my-nginx
  19.         image: nginx
  20.         ports:
  21.         - containerPort: 80
  22. EOF
  24. # Create a kustomization.yaml
  25. cat <<EOF >./kustomization.yaml
  26. namePrefix: dev-
  27. commonLabels:
  28.   app: my-nginx
  29. resources:
  30. - deployment.yaml
  31. EOF

Run the following command to apply the Deployment object dev-my-nginx:

  1. > kubectl apply -k ./
  2. deployment.apps/dev-my-nginx created

Run one of the following commands to view the Deployment object dev-my-nginx:

  1. kubectl get -k ./
  1. kubectl describe -k ./

Run the following command to compare the Deployment object dev-my-nginx against the state that the cluster would be in if the manifest was applied:

  1. kubectl diff -k ./

Run the following command to delete the Deployment object dev-my-nginx:

  1. > kubectl delete -k ./
  2. deployment.apps "dev-my-nginx" deleted

Kustomize Feature List

FieldTypeExplanation
namespacestringadd namespace to all resources
namePrefixstringvalue of this field is prepended to the names of all resources
nameSuffixstringvalue of this field is appended to the names of all resources
commonLabelsmap[string]stringlabels to add to all resources and selectors
commonAnnotationsmap[string]stringannotations to add to all resources
resources[]stringeach entry in this list must resolve to an existing resource configuration file
configmapGenerator[]ConfigMapArgsEach entry in this list generates a ConfigMap
secretGenerator[]SecretArgsEach entry in this list generates a Secret
generatorOptionsGeneratorOptionsModify behaviors of all ConfigMap and Secret generator
bases[]stringEach entry in this list should resolve to a directory containing a kustomization.yaml file
patchesStrategicMerge[]stringEach entry in this list should resolve a strategic merge patch of a Kubernetes object
patchesJson6902[]Json6902Each entry in this list should resolve to a Kubernetes object and a Json Patch
vars[]VarEach entry is to capture text from one resource’s field
images[]ImageEach entry is to modify the name, tags and/or digest for one image without creating patches
configurations[]stringEach entry in this list should resolve to a file containing Kustomize transformer configurations
crds[]stringEach entry in this list should resolve to an OpenAPI definition file for Kubernetes types

What’s next

Feedback

Was this page helpful?

Thanks for the feedback. If you have a specific, answerable question about how to use Kubernetes, ask it on Stack Overflow. Open an issue in the GitHub repo if you want to report a problem or suggest an improvement.