Cluster Templating

The command kops replace can replace a cluster desired configuration from the config in a yaml file (see cli/kops_replace.md).

It is possible to generate that yaml file from a template, using the command kops toolbox template (see cli/kops_toolbox_template.md).

This document details the template language used.

The file passed as --template must be a go template. Example:

  1. # File cluster.tmpl.yaml
  2. apiVersion: kops.k8s.io/v1alpha2
  3. kind: InstanceGroup
  4. metadata:
  5.   labels:
  6.   kops.k8s.io/cluster: {{.clusterName}}.{{.dnsZone}}
  7.   name: nodes
  8. spec:
  9.   image: {{ ChannelRecommendedImage .cloud .kubernetesVersion .architecture }}
  10.   kubernetesVersion: {{ ChannelRecommendedKubernetesUpgradeVersion .kubernetesVersion }}
  11.   machineType: m4.large
  12.   maxPrice: "0.5"
  13.   maxSize: 20
  14.   minSize: 15
  15.   role: Node
  16.   rootVolumeSize: 100
  17.   subnets:
  18.   - {{.awsRegion}}a
  19.   - {{.awsRegion}}b
  20.   - {{.awsRegion}}c

You can pass configuration such as an environment file by using the --values PATH command line option. Note --values is a slice so can be defined multiple times; the configuration is overridden by each configuration file (so order is important assuming duplicating values); a use-case for this would be a default configuration which upstream clusters can override.

The file passed as --values must contain the variables referenced in the template. Example:

  1. # File values.yaml
  2. clusterName: eu1
  3. kubernetesVersion: 1.7.1
  4. dnsZone: k8s.example.com
  5. awsRegion: eu-west-1

When multiple environment files are passed using --values kOps performs a deep merge, for example given the following two files:

  1. # File values-a.yaml
  2. instanceGroups:
  3.   foo:
  4.     ami: ami-1234567
  5.     type: m4.large
  7. # File values-b.yaml
  8. instanceGroups:
  9.   foo:
  10.     type: t2.large

Would result in the instanceGroups.foo object having two properties: {"ami": "ami-1234567", "type": "t2.large"}.

Besides specifying values through an environment file it is also possible to pass variables directly on the command line using the --set and --set-string command line options. The difference between the two options is that --set-string will always yield a string value while --set will cause the value to be parsed as a YAML value, for example the value true would turn into a boolean with --set while with --set-string it will be the literal string "true". The format for specifying a variable is as follows:

  1. kops toolbox template --template mytemplate.tpl --set 'version=1.0,foo.bar=baz' --set-string 'foo.myArray={1,2,3}' --set 'foo.myArray[1]=false,foo.myArray[3]=4'

which would yield the same values as using the --values option with the following file

  1. version: 1.0
  2. foo:
  3.   bar: baz
  4.   myArray:
  5.   - "1"
  6.   - false
  7.   - "3"
  8.   - 4

Running kops toolbox template replaces the placeholders in the template by values and generates the file output.yaml, which can then be used to replace the desired cluster configuration with kops replace -f cluster.yaml.

Note: when creating a cluster desired configuration template, you can

  • use kops get k8s-cluster.example.com -o yaml > cluster-desired-config.yaml to create the cluster desired configuration file (see cli/kops_get.md). The values in this file are defined in cluster_spec.md.
  • replace values by placeholders in that file to create the template.

Templates

The --template command line option can point to either a specific file or a directory with a collection of templates. An example usage would be;

  1. $ kops toolbox template --values dev.yaml --template cluster.yaml --template instance_group_directory

The cluster.yaml (your main cluster spec for example) would be written first followed by any templates found in the instance_group_directory directory. Note the toolbox will automatically add YAML separators between the documents for you.

Snippets

The toolbox template also supports the reuse or break up of code blocks into snippets directories. By passing a --snippets PATH to a directory holding templates;

  1. $ kops toolbox template --values dev.yaml --template cluster.yaml --template instancegroups --snippets snippets

The example below assumes you have placed the appropriate files i.e. (nodes.json, master.json etc) in to the snippets directory. Note, the namespace of the snippets are flat and always the basename() of the file path; so snippets/components/docker.options is still referred to as ‘docker.options’.

  1. apiVersion: kops.k8s.io/v1alpha2
  2. kind: Cluster
  3. metadata:
  4.   name: {{ .environment }}.{{ .dns_zone }}
  5. spec:
  6.   docker:
  7.     {{ include "docker" . | indent 4 }}
  8.   additionalPolicies:
  9.     master: |
  10.       {{ include "masters.json" . | indent 6 }}
  11.     node: |
  12.       {{ include "nodes.json" . | indent 6 }}

Template Functions

Kops specific functions

ChannelRecommendedKopsKubernetesVersion

This function returns the kubernetes version recommended for the running kops version.

ChannelRecommendedKubernetesUpgradeVersion

This function returns the recommended kubernetes version given that you currently run <kubernetesVersion>. Typically this is the latest patch version supported by the given channel.

ChannelRecommendedImage

This function returns the recommended image for the given cloud provider and kubernetes version.

Sprig functions

The entire set of Sprig functions are available within the templates for you. Note if you want to use the ‘defaults’ functions switch off the verification check on the command line by --fail-on-missing=false;

  1. image: {{ default $image $node.image }}
  2. machineType: {{ default $instance $node.machine_type }}
  3. maxSize: {{ default "10" $node.max_size }}
  4. minSize: {{ default "1" $node.min_size }}

Assigning entire arrays is also supported with Sprig’s toJson function.

  1. # template
  2. spec:
  3.   kubernetesApiAccess: {{.allowedIPs | toJson }}
  1. # values
  2. allowedIPs:
  3. - 1.2.3.4/32
  4. - 4.3.2.1/32
  1. # rendered
  2. spec:
  3.   kubernetesApiAccess: ["1.2.3.4/32","4.3.2.1/32"]

Formatting

Formatting in golang templates is a pain! At the start or at the end of a statement can be infuriating to get right, so a --format-yaml=true (defaults to false) command line option has been added. This will first unmarshal the generated content (performing a syntax verification) and then marshal back the content removing all those nasty formatting issues, newlines etc.