Configuration Options

k0s Control plane can be configured via a YAML config file. By default k0s controller command reads a file called k0s.yaml but can be told to read any yaml file via --config option.

Configuration file reference

Note: Many of the options configure things deep down in the “stack” on various components. So please make sure you understand what is being configured and whether or not it works in your specific environment.

A full config file with defaults generated by the k0s default-config command:

  1. apiVersion: k0s.k0sproject.io/v1beta1
  2. kind: Cluster
  3. metadata:
  4.   name: k0s
  5. spec:
  6.   api:
  7.     port: 6443
  8.     k0sApiPort: 9443
  9.     externalAddress: my-lb-address.example.com
  10.     address: 192.168.68.104
  11.     sans:
  12.       - 192.168.68.104
  13.   storage:
  14.     type: etcd
  15.     etcd:
  16.       peerAddress: 192.168.68.104
  17.   network:
  18.     podCIDR: 10.244.0.0/16
  19.     serviceCIDR: 10.96.0.0/12
  20.     provider: calico
  21.     calico:
  22.       mode: vxlan
  23.       vxlanPort: 4789
  24.       vxlanVNI: 4096
  25.       mtu: 1450
  26.       wireguard: false
  27.       flexVolumeDriverPath: /usr/libexec/k0s/kubelet-plugins/volume/exec/nodeagent~uds
  28.       withWindowsNodes: false
  29.       overlay: Always
  30.   podSecurityPolicy:
  31.     defaultPolicy: 00-k0s-privileged
  32.   telemetry:
  33.     interval: 10m0s
  34.     enabled: true
  35.   installConfig:
  36.     users:
  37.       etcdUser: etcd
  38.       kineUser: kube-apiserver
  39.       konnectivityUser: konnectivity-server
  40.       kubeAPIserverUser: kube-apiserver
  41.       kubeSchedulerUser: kube-scheduler
  42.   images:
  43.     konnectivity:
  44.       image: us.gcr.io/k8s-artifacts-prod/kas-network-proxy/proxy-agent
  45.       version: v0.0.13
  46.     metricsserver:
  47.       image: gcr.io/k8s-staging-metrics-server/metrics-server
  48.       version: v0.3.7
  49.     kubeproxy:
  50.       image: k8s.gcr.io/kube-proxy
  51.       version: v1.20.5
  52.     coredns:
  53.       image: docker.io/coredns/coredns
  54.       version: 1.7.0
  55.     calico:
  56.       cni:
  57.         image: calico/cni
  58.         version: v3.16.2
  59.       flexvolume:
  60.         image: calico/pod2daemon-flexvol
  61.         version: v3.16.2
  62.       node:
  63.         image: calico/node
  64.         version: v3.16.2
  65.       kubecontrollers:
  66.         image: calico/kube-controllers
  67.         version: v3.16.2
  68.   konnectivity:
  69.     agentPort: 8132
  70.     adminPort: 8133

spec.api

  • externalAddress: If k0s controllers are running behind a loadbalancer provide the loadbalancer address here. This will configure all cluster components to connect to this address and also configures this address to be used when joining new nodes into the cluster.
  • address: The local address to bind API on. Also used as one of the addresses pushed on the k0s create service certificate on the API. Defaults to first non-local address found on the node.
  • sans: List of additional addresses to push to API servers serving certificate
  • extraArgs: Map of key-values (strings) for any extra arguments you wish to pass down to Kubernetes api-server process
  • port: custom port for kube-api server to listen on (default value 6443)
  • k0sApiPort: custom port for k0s-api server to listen on (default value 9443)

Keep in mind, in case if port and k0sApiPort are used with externalAddress setting, the LB serving at externalAddress must listen on the same ports.

spec.controllerManager

  • extraArgs: Map of key-values (strings) for any extra arguments you wish to pass down to Kubernetes controller manager process

spec.scheduler

  • extraArgs: Map of key-values (strings) for any extra arguments you wish to pass down to Kubernetes scheduler process

spec.storage

  • type: Type of the data store, either etcd or kine.
  • etcd.peerAddress: Nodes address to be used for etcd cluster peering.
  • kine.dataSource: kine datasource URL.

Using type etcd will make k0s to create and manage an elastic etcd cluster within the controller nodes.

spec.network

  • provider: Network provider, either calico, kuberouter or custom. In case of custom user can push any network provider. (default kuberouter)
  • podCIDR: Pod network CIDR to be used in the cluster
  • serviceCIDR: Network CIDR to be used for cluster VIP services.

Note: In case of custom network it’s fully in users responsibility to configure ALL the CNI related setups. This includes the CNI provider itself plus all the host levels setups it might need such as CNI binaries.

Note: After the cluster has been initialized with one network provider, it is not currently supported to change the network provider.

spec.network.calico

  • mode: vxlan (default) or ipip
  • vxlanPort: The UDP port to use for VXLAN (default 4789)
  • vxlanVNI: The virtual network ID to use for VXLAN. (default: 4096)
  • mtu: MTU to use for overlay network (default 1450)
  • wireguard: enable wireguard based encryption (default false). Your host system must be wireguard ready. See https://docs.projectcalico.org/security/encrypt-cluster-pod-traffic for details.
  • flexVolumeDriverPath: The host path to use for Calicos flex-volume-driver (default: /usr/libexec/k0s/kubelet-plugins/volume/exec/nodeagent~uds). This should only need to be changed if the default path is unwriteable. See https://github.com/projectcalico/calico/issues/2712 for details. This option should ideally be paired with a custom volumePluginDir in the profile used on your worker nodes.
  • ipAutodetectionMethod: To force non-default behaviour for Calico to pick up the interface for pod network inter-node routing. (default "", i.e. not set so Calico will use it’s own defaults) See more at: https://docs.projectcalico.org/reference/node/configuration#ip-autodetection-methods

spec.network.kuberouter

  • autoMTU: Autodetection of used MTU (default true)
  • mtu: Override MTU setting, if set autoMTU needs to be set to false
  • peerRouterIPs: comma separated list of global peer addresses
  • peerRouterASNs: comma separated list of global peer ASNs

Kube-router allows many aspects of the networking to be configured per node, service and pod. For further details see kube-router user guide

spec.podSecurityPolicy

Configures the default psp to be set. k0s creates two PSPs out of box:

  • 00-k0s-privileged (default): no restrictions, always also used for Kubernetes/k0s level system pods
  • 99-k0s-restricted: no host namespaces or root users allowed, no bind mounts from host

As a user you can of course create any supplemental PSPs and bind them to users / access accounts as you need.

spec.workerProfiles

Array of spec.workerProfiles.workerProfile Each element has following properties: - name: string, name, used as profile selector for the worker process - values: mapping object

For each profile the control plane will create separate ConfigMap with kubelet-config yaml. Based on the --profile argument given to the k0s worker the corresponding ConfigMap would be used to extract kubelet-config.yaml from. values are recursively merged with default kubelet-config.yaml

There are a few fields that cannot be overridden: - clusterDNS - clusterDomain - apiVersion - kind

Example:

  1. spec:
  2.   workerProfiles:
  3.     - name: custom-role
  4.       values:
  5.          key: value
  6.          mapping:
  7.              innerKey: innerValue

Custom volumePluginDir:

  1. spec:
  2.   workerProfiles:
  3.     - name: custom-role
  4.       values:
  5.          volumePluginDir: /var/libexec/k0s/kubelet-plugins/volume/exec

spec.images

Each node under the images key has the same structure

  1. spec:
  2.   images:
  3.     konnectivity:
  4.       image: calico/kube-controllers
  5.       version: v3.16.2

Following keys are available:

  • spec.images.konnectivity
  • spec.images.metricsserver
  • spec.images.kubeproxy
  • spec.images.coredns
  • spec.images.calico.cni
  • spec.images.calico.flexvolume
  • spec.images.calico.node
  • spec.images.calico.kubecontrollers
  • spec.images.repository

If spec.images.repository is set and not empty, every image will be pulled from images.repository If spec.images.default_pull_policy is set ant not empty, it will be used as a pull policy for each bundled image.

Example:

  1. images:
  2.   repository: "my.own.repo"
  3.   konnectivity:
  4.     image: calico/kube-controllers
  5.     version: v3.16.2
  6.   metricsserver:
  7.     image: gcr.io/k8s-staging-metrics-server/metrics-server
  8.     version: v0.3.7

In the runtime the image names will be calculated as my.own.repo/calico/kube-controllers:v3.16.2 and my.own.repo/k8s-staging-metrics-server/metrics-server.

This only affects the location where images are getting pulled, omitting an image specification here will not disable the component from being deployed.

spec.extensions.helm

List of Helm repositories and charts to deploy during cluster bootstrap. For more information, see Helm Charts.

spec.konnectivity

Konnectivity related settings

  • agentPort agent port to listen on (default 8132)
  • adminPort admin port to listen on (default 8133)

Telemetry

To build better end user experience we collect and send telemetry data from clusters. It is enabled by default and can be disabled by settings corresponding option as false The default interval is 10 minutes, any valid value for time.Duration string representation can be used as a value. Example

  1. spec:
  2.     telemetry:
  3.       interval: 2m0s
  4.       enabled: true

Configuration Validation

k0s command-line interface has the ability to validate config syntax:

  1. $ k0s validate config --config path/to/config/file

validate config sub-command can validate the following:

  1. YAML formatting
  2. SANs addresses
  3. Network providers
  4. Worker profiles