- kubeadm Configuration (v1beta4)
- Migration from old kubeadm config versions
- Basics
- Kubeadm init configuration types
- Kubeadm join configuration types
- Kubeadm reset configuration types
- Kubeadm upgrade configuration types
- Resource Types
BootstrapToken
BootstrapTokenString
ClusterConfiguration
InitConfiguration
JoinConfiguration
ResetConfiguration
UpgradeConfiguration
APIEndpoint
APIServer
Arg
BootstrapTokenDiscovery
ControlPlaneComponent
DNS
Discovery
EncryptionAlgorithmType
EnvVar
Etcd
ExternalEtcd
FileDiscovery
HostPathMount
ImageMeta
JoinControlPlane
LocalEtcd
Networking
NodeRegistrationOptions
Patches
Proxy
Timeouts
UpgradeApplyConfiguration
UpgradeDiffConfiguration
UpgradeNodeConfiguration
UpgradePlanConfiguration
kubeadm Configuration (v1beta4)
Overview
Package v1beta4 defines the v1beta4 version of the kubeadm configuration file format. This version improves on the v1beta3 format by fixing some minor issues and adding a few new fields.
A list of changes since v1beta3:
- TODO https://github.com/kubernetes/kubeadm/issues/2890
- Support custom environment variables in control plane components under
ClusterConfiguration
. UseapiServer.extraEnvs
,controllerManager.extraEnvs
,scheduler.extraEnvs
,etcd.local.extraEnvs
. - The
ResetConfiguration
API type is now supported in v1beta4. Users are able to reset a node by passing a--config
file tokubeadm reset
. - Dry run mode is now configureable in InitConfiguration and JoinConfiguration.
- Replace the existing string/string extra argument maps with structured extra arguments that support duplicates. The change applies to
ClusterConfiguration
-apiServer.extraArgs
,controllerManager.extraArgs
,scheduler.extraArgs
,etcd.local.extraArgs
. Also tonodeRegistration.kubeletExtraArgs
. - Add
ClusterConfiguration.encryptionAlgorithm
that can be used to set the asymmetric encryption algorithm used for this cluster’s keys and certificates. Can be one of"RSA-2048"
(default),"RSA-3072"
,"RSA-4096"
or"ECDSA-P256"
. - Add
ClusterConfiguration.dns.disabled
andClusterConfiguration.proxy.disabled
that can be used to disable the CoreDNS and kube-proxy addons during cluster initialization. Skipping the related addons phases, during cluster creation will set the same fields tofalse
. - Add the
nodeRegistration.imagePullSerial
field inInitConfiguration
andJoinConfiguration
, which can be used to control if kubeadm pulls images serially or in parallel. - The
UpgradeConfiguration
kubeadm API is now supported in v1beta4 when passing--config
tokubeadm upgrade
subcommands. Usage of component configuration forkubelet
andkube-proxy
,InitConfiguration
andClusterConfiguration
is deprecated and will be ignored when passing--config
toupgrade
subcommands. - Add a
Timeouts
structure toInitConfiguration
,JoinConfiguration
,ResetConfiguration
andUpgradeConfiguration
that can be used to configure various timeouts. - Add a
certificateValidityPeriod
andcaCertificateValidityPeriod
fields toClusterConfiguration
. These fields can be used to control the validity period of certificates generated by kubeadm during sub-commands such asinit
,join
,upgrade
andcerts
. Default values continue to be 1 year for non-CA certificates and 10 years for CA certificates. Only non-CA certificates continue to be renewable bykubeadm certs renew
.
Migration from old kubeadm config versions
- kubeadm v1.15.x and newer can be used to migrate from v1beta1 to v1beta2.
- kubeadm v1.22.x and newer no longer support v1beta1 and older APIs, but can be used to migrate v1beta2 to v1beta3.
- kubeadm v1.27.x and newer no longer support v1beta2 and older APIs.
- TODO: https://github.com/kubernetes/kubeadm/issues/2890 add version that can be used to convert to v1beta4
Basics
The preferred way to configure kubeadm is to pass an YAML configuration file with the `—config“ option. Some of the configuration options defined in the kubeadm config file are also available as command line flags, but only the most common/simple use case are supported with this approach.
A kubeadm config file could contain multiple configuration types separated using three dashes (---
).
kubeadm supports the following configuration types:
apiVersion: kubeadm.k8s.io/v1beta4
kind: InitConfiguration
apiVersion: kubeadm.k8s.io/v1beta4
kind: ClusterConfiguration
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
apiVersion: kubeadm.k8s.io/v1beta4
kind: JoinConfiguration
apiVersion: kubeadm.k8s.io/v1beta4
kind: ResetConfiguration
To print the defaults for “init” and “join” actions use the following commands:
kubeadm config print init-defaults
kubeadm config print join-defaults
kubeadm config print reset-defaults
kubeadm config print upgrade-defaults
The list of configuration types that must be included in a configuration file depends by the action you are performing (init or
join`) and by the configuration options you are going to use (defaults or advanced customization).
If some configuration types are not provided, or provided only partially, kubeadm will use default values; defaults provided by kubeadm includes also enforcing consistency of values across components when required (e.g. --cluster-cidr
flag on controller manager and clusterCIDR
on kube-proxy).
Users are always allowed to override default values, with the only exception of a small subset of setting with relevance for security (e.g. enforce authorization-mode Node and RBAC on api server).
If the user provides a configuration types that is not expected for the action you are performing, kubeadm will ignore those types and print a warning.
Kubeadm init configuration types
When executing kubeadm init with the `—config“ option, the following configuration types could be used: InitConfiguration, ClusterConfiguration, KubeProxyConfiguration, KubeletConfiguration, but only one between InitConfiguration and ClusterConfiguration is mandatory.
apiVersion: kubeadm.k8s.io/v1beta4
kind: InitConfiguration
bootstrapTokens:
...
nodeRegistration:
...
The InitConfiguration type should be used to configure runtime settings, that in case of kubeadm init are the configuration of the bootstrap token and all the setting which are specific to the node where kubeadm is executed, including:
NodeRegistration, that holds fields that relate to registering the new node to the cluster; use it to customize the node name, the CRI socket to use or any other settings that should apply to this node only (e.g. the node ip).
LocalAPIEndpoint, that represents the endpoint of the instance of the API server to be deployed on this node; use it e.g. to customize the API server advertise address.
apiVersion: kubeadm.k8s.io/v1beta4
kind: ClusterConfiguration
networking:
...
etcd:
...
apiServer:
extraArgs:
...
extraVolumes:
...
...
The ClusterConfiguration type should be used to configure cluster-wide settings, including settings for:
networking
that holds configuration for the networking topology of the cluster; use it e.g. to customize Pod subnet or services subnet.etcd
: use it e.g. to customize the local etcd or to configure the API server for using an external etcd cluster.kube-apiserver, kube-scheduler, kube-controller-manager configurations; use it to customize control-plane components by adding customized setting or overriding kubeadm default settings.
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
...
The KubeProxyConfiguration type should be used to change the configuration passed to kube-proxy instances deployed in the cluster. If this object is not provided or provided only partially, kubeadm applies defaults.
See https://kubernetes.io/docs/reference/command-line-tools-reference/kube-proxy/ or https://pkg.go.dev/k8s.io/kube-proxy/config/v1alpha1#KubeProxyConfiguration for kube-proxy official documentation.
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
...
The KubeletConfiguration type should be used to change the configurations that will be passed to all kubelet instances deployed in the cluster. If this object is not provided or provided only partially, kubeadm applies defaults.
See https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/ or https://pkg.go.dev/k8s.io/kubelet/config/v1beta1#KubeletConfiguration for kubelet official documentation.
Here is a fully populated example of a single YAML file containing multiple configuration types to be used during a kubeadm init
run.
apiVersion: kubeadm.k8s.io/v1beta4
kind: InitConfiguration
bootstrapTokens:
- token: "9a08jv.c0izixklcxtmnze7"
description: "kubeadm bootstrap token"
ttl: "24h"
- token: "783bde.3f89s0fje9f38fhf"
description: "another bootstrap token"
usages:
- authentication
- signing
groups:
- system:bootstrappers:kubeadm:default-node-token
nodeRegistration:
name: "ec2-10-100-0-1"
criSocket: "unix:///var/run/containerd/containerd.sock"
taints:
- key: "kubeadmNode"
value: "someValue"
effect: "NoSchedule"
kubeletExtraArgs:
- name: v
value: "5"
ignorePreflightErrors:
- IsPrivilegedUser
imagePullPolicy: "IfNotPresent"
imagePullSerial: true
localAPIEndpoint:
advertiseAddress: "10.100.0.1"
bindPort: 6443
certificateKey: "e6a2eb8581237ab72a4f494f30285ec12a9694d750b9785706a83bfcbbbd2204"
skipPhases:
- preflight
timeouts:
controlPlaneComponentHealthCheck: "60s"
kubenetesAPICall: "40s"
---
apiVersion: kubeadm.k8s.io/v1beta4
kind: ClusterConfiguration
etcd:
# one of local or external
local:
imageRepository: "registry.k8s.io"
imageTag: "3.2.24"
dataDir: "/var/lib/etcd"
extraArgs:
- name: listen-client-urls
value: http://10.100.0.1:2379
extraEnvs:
- name: SOME_VAR
value: SOME_VALUE
serverCertSANs:
- "ec2-10-100-0-1.compute-1.amazonaws.com"
peerCertSANs:
- "10.100.0.1"
# external:
# endpoints:
# - "10.100.0.1:2379"
# - "10.100.0.2:2379"
# caFile: "/etcd/kubernetes/pki/etcd/etcd-ca.crt"
# certFile: "/etcd/kubernetes/pki/etcd/etcd.crt"
# keyFile: "/etcd/kubernetes/pki/etcd/etcd.key"
networking:
serviceSubnet: "10.96.0.0/16"
podSubnet: "10.244.0.0/24"
dnsDomain: "cluster.local"
kubernetesVersion: "v1.21.0"
controlPlaneEndpoint: "10.100.0.1:6443"
apiServer:
extraArgs:
- name: authorization-mode
value: "Node,RBAC"
extraEnvs:
- name: SOME_VAR
value: SOME_VALUE
extraVolumes:
- name: "some-volume"
hostPath: "/etc/some-path"
mountPath: "/etc/some-pod-path"
readOnly: false
pathType: File
certSANs:
- "10.100.1.1"
- "ec2-10-100-0-1.compute-1.amazonaws.com"
controllerManager:
extraArgs:
- name: node-cidr-mask-size
value: "20"
extraVolumes:
- name: "some-volume"
hostPath: "/etc/some-path"
mountPath: "/etc/some-pod-path"
readOnly: false
pathType: File
scheduler:
extraArgs:
- name: address
value: "10.100.0.1"
extraVolumes:
- name: "some-volume"
hostPath: "/etc/some-path"
mountPath: "/etc/some-pod-path"
readOnly: false
pathType: File
certificatesDir: "/etc/kubernetes/pki"
imageRepository: "registry.k8s.io"
clusterName: "example-cluster"
encryptionAlgorithm: ECDSA-P256
dns:
disabled: true # disable CoreDNS
proxy:
diabled: true # disable kube-proxy
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
# kubelet specific options here
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
# kube-proxy specific options here
Kubeadm join configuration types
When executing kubeadm join with the —config option, the JoinConfiguration type should be provided.
apiVersion: kubeadm.k8s.io/v1beta4
kind: JoinConfiguration
discovery:
bootstrapToken:
apiServerEndpoint: some-address:6443
token: abcdef.0123456789abcdef
unsafeSkipCAVerification: true
tlsBootstrapToken: abcdef.0123456789abcdef
The JoinConfiguration type should be used to configure runtime settings, that in case of kubeadm join are the discovery method used for accessing the cluster info and all the setting which are specific to the node where kubeadm is executed, including:
nodeRegistration
, that holds fields that relate to registering the new node to the cluster; use it to customize the node name, the CRI socket to use or any other settings that should apply to this node only (e.g. the node ip).apiEndpoint
, that represents the endpoint of the instance of the API server to be eventually deployed on this node.
Kubeadm reset configuration types
When executing kubeadm reset
with the --config
option, the ResetConfiguration
type should be provided.
apiVersion: kubeadm.k8s.io/v1beta4
kind: ResetConfiguration
...
Kubeadm upgrade configuration types
When executing kubeadm upgrade
with the --config
option, the UpgradeConfiguration
type should be provided.
apiVersion: kubeadm.k8s.io/v1beta4
kind: UpgradeConfiguration
apply:
...
diff:
...
node:
...
plan:
...
The UpgradeConfiguration
structure includes a few substructures that only apply to different subcommands of kubeadm upgrade
. For example, the apply
substructure will be used with the kubeadm upgrade apply
subcommand and all other substructures will be ignored in such a case.
Resource Types
BootstrapToken
Appears in:
BootstrapToken describes one bootstrap token, stored as a Secret in the cluster
Field | Description |
---|---|
token [Required]BootstrapTokenString |
|
description string |
|
ttl meta/v1.Duration |
|
expires meta/v1.Time |
|
usages []string |
|
groups []string |
|
BootstrapTokenString
Appears in:
BootstrapTokenString is a token of the format abcdef.abcdef0123456789
that is used for both validation of the practically of the API server from a joining node’s point of view and as an authentication method for the node in the bootstrap phase of “kubeadm join”. This token is and should be short-lived.
Field | Description |
---|---|
- [Required]string | No description provided. |
- [Required]string | No description provided. |
ClusterConfiguration
ClusterConfiguration contains cluster-wide configuration for a kubeadm cluster.
Field | Description |
---|---|
apiVersion string | kubeadm.k8s.io/v1beta4 |
kind string | ClusterConfiguration |
etcd Etcd |
|
networking Networking |
|
kubernetesVersion string |
|
controlPlaneEndpoint string |
|
apiServer APIServer |
|
controllerManager ControlPlaneComponent |
|
scheduler ControlPlaneComponent |
|
dns DNS |
|
proxy [Required]Proxy |
|
certificatesDir string |
|
imageRepository string |
|
featureGates map[string]bool |
|
clusterName string | The cluster name. |
encryptionAlgorithm EncryptionAlgorithmType |
|
certificateValidityPeriod meta/v1.Duration |
|
caCertificateValidityPeriod meta/v1.Duration |
|
InitConfiguration
InitConfiguration contains a list of elements that is specific “kubeadm init”-only runtime information. kubeadm init
-only information. These fields are solely used the first time kubeadm init
runs. After that, the information in the fields IS NOT uploaded to the kubeadm-config
ConfigMap that is used by kubeadm upgrade
for instance. These fields must be omitempty.
Field | Description |
---|---|
apiVersion string | kubeadm.k8s.io/v1beta4 |
kind string | InitConfiguration |
bootstrapTokens []BootstrapToken |
|
dryRun [Required]bool |
|
nodeRegistration NodeRegistrationOptions |
|
localAPIEndpoint APIEndpoint |
|
certificateKey string |
|
skipPhases []string |
|
patches Patches |
|
timeouts Timeouts |
|
JoinConfiguration
JoinConfiguration contains elements describing a particular node.
Field | Description |
---|---|
apiVersion string | kubeadm.k8s.io/v1beta4 |
kind string | JoinConfiguration |
dryRun bool |
|
nodeRegistration NodeRegistrationOptions |
|
caCertPath string |
|
discovery [Required]Discovery |
|
controlPlane JoinControlPlane |
|
skipPhases []string |
|
patches Patches |
|
timeouts Timeouts |
|
ResetConfiguration
ResetConfiguration contains a list of fields that are specifically kubeadm reset
-only runtime information.
Field | Description |
---|---|
apiVersion string | kubeadm.k8s.io/v1beta4 |
kind string | ResetConfiguration |
cleanupTmpDir bool |
|
certificatesDir string |
|
criSocket string |
|
dryRun bool |
|
force bool | The |
ignorePreflightErrors []string |
|
skipPhases []string |
|
unmountFlags []string |
|
timeouts Timeouts | Timeouts holds various timeouts that apply to kubeadm commands. |
UpgradeConfiguration
UpgradeConfiguration contains a list of options that are specific to kubeadm upgrade
subcommands.
Field | Description |
---|---|
apiVersion string | kubeadm.k8s.io/v1beta4 |
kind string | UpgradeConfiguration |
apply UpgradeApplyConfiguration |
|
diff UpgradeDiffConfiguration |
|
node UpgradeNodeConfiguration |
|
plan UpgradePlanConfiguration |
|
timeouts Timeouts |
|
APIEndpoint
Appears in:
APIEndpoint struct contains elements of API server instance deployed on a node.
Field | Description |
---|---|
advertiseAddress string |
|
bindPort int32 |
|
APIServer
Appears in:
APIServer holds settings necessary for API server deployments in the cluster
Field | Description |
---|---|
ControlPlaneComponent [Required]ControlPlaneComponent | (Members of ControlPlaneComponent are embedded into this type.) No description provided. |
certSANs []string |
|
Arg
Appears in:
Arg represents an argument with a name and a value.
Field | Description |
---|---|
name [Required]string | The name of the argument. |
value [Required]string | The value of the argument. |
BootstrapTokenDiscovery
Appears in:
BootstrapTokenDiscovery is used to set the options for bootstrap token based discovery.
Field | Description |
---|---|
token [Required]string |
|
apiServerEndpoint string |
|
caCertHashes []string |
|
unsafeSkipCAVerification bool |
|
ControlPlaneComponent
Appears in:
ControlPlaneComponent holds settings common to control plane component of the cluster
Field | Description |
---|---|
extraArgs []Arg |
|
extraVolumes []HostPathMount |
|
extraEnvs []EnvVar |
|
DNS
Appears in:
DNS defines the DNS addon that should be used in the cluster
Field | Description |
---|---|
ImageMeta [Required]ImageMeta | (Members of ImageMeta are embedded into this type.)
|
disabled [Required]bool |
|
Discovery
Appears in:
Discovery specifies the options for the kubelet to use during the TLS Bootstrap process
Field | Description |
---|---|
bootstrapToken BootstrapTokenDiscovery |
|
file FileDiscovery |
|
tlsBootstrapToken string |
|
EncryptionAlgorithmType
(Alias of string
)
Appears in:
EncryptionAlgorithmType can define an asymmetric encryption algorithm type.
EnvVar
Appears in:
EnvVar represents an environment variable present in a Container.
Field | Description |
---|---|
EnvVar [Required]core/v1.EnvVar | (Members of EnvVar are embedded into this type.) No description provided. |
Etcd
Appears in:
Etcd contains elements describing Etcd configuration.
Field | Description |
---|---|
local LocalEtcd |
|
external ExternalEtcd |
|
ExternalEtcd
Appears in:
ExternalEtcd describes an external etcd cluster. Kubeadm has no knowledge of where certificate files live and they must be supplied.
Field | Description |
---|---|
endpoints [Required][]string |
|
caFile [Required]string |
|
certFile [Required]string |
|
keyFile [Required]string |
|
FileDiscovery
Appears in:
FileDiscovery is used to specify a file or URL to a kubeconfig file from which to load cluster information.
Field | Description |
---|---|
kubeConfigPath [Required]string |
|
HostPathMount
Appears in:
HostPathMount contains elements describing volumes that are mounted from the host.
Field | Description |
---|---|
name [Required]string |
|
hostPath [Required]string |
|
mountPath [Required]string |
|
readOnly bool |
|
pathType core/v1.HostPathType |
|
ImageMeta
Appears in:
ImageMeta allows to customize the image used for components that are not originated from the Kubernetes/Kubernetes release process
Field | Description |
---|---|
imageRepository string |
|
imageTag string |
|
JoinControlPlane
Appears in:
JoinControlPlane contains elements describing an additional control plane instance to be deployed on the joining node.
Field | Description |
---|---|
localAPIEndpoint APIEndpoint |
|
certificateKey string |
|
LocalEtcd
Appears in:
LocalEtcd describes that kubeadm should run an etcd cluster locally.
Field | Description |
---|---|
ImageMeta [Required]ImageMeta | (Members of ImageMeta are embedded into this type.)ImageMeta allows to customize the container used for etcd |
dataDir [Required]string |
|
extraArgs [Required][]Arg |
|
extraEnvs []EnvVar |
|
serverCertSANs []string |
|
peerCertSANs []string |
|
Networking
Appears in:
Networking contains elements describing cluster’s networking configuration.
Field | Description |
---|---|
serviceSubnet string |
|
podSubnet string |
|
dnsDomain string |
|
NodeRegistrationOptions
Appears in:
NodeRegistrationOptions holds fields that relate to registering a new control-plane or node to the cluster, either via kubeadm init
or kubeadm join
.
Field | Description |
---|---|
name string |
|
criSocket string |
|
taints [Required][]core/v1.Taint |
|
kubeletExtraArgs []Arg |
|
ignorePreflightErrors []string |
|
imagePullPolicy core/v1.PullPolicy |
|
imagePullSerial bool |
|
Patches
Appears in:
Patches contains options related to applying patches to components deployed by kubeadm.
Field | Description |
---|---|
directory string |
|
Proxy
Appears in:
Proxy defines the proxy addon that should be used in the cluster.
Field | Description |
---|---|
disabled [Required]bool |
|
Timeouts
Appears in:
Timeouts holds various timeouts that apply to kubeadm commands.
Field | Description |
---|---|
controlPlaneComponentHealthCheck meta/v1.Duration |
|
kubeletHealthCheck meta/v1.Duration |
|
kubernetesAPICall meta/v1.Duration |
|
etcdAPICall meta/v1.Duration |
|
tlsBootstrap meta/v1.Duration |
|
discovery meta/v1.Duration |
|
upgradeManifests [Required]meta/v1.Duration |
|
UpgradeApplyConfiguration
Appears in:
UpgradeApplyConfiguration contains a list of configurable options which are specific to the “kubeadm upgrade apply” command.
Field | Description |
---|---|
kubernetesVersion string |
|
allowExperimentalUpgrades bool |
|
allowRCUpgrades bool | Enable |
certificateRenewal bool |
|
dryRun bool |
|
etcdUpgrade bool |
|
forceUpgrade bool |
|
ignorePreflightErrors []string |
|
patches Patches |
|
printConfig bool |
|
skipPhases [Required][]string |
|
imagePullPolicy core/v1.PullPolicy |
|
imagePullSerial bool |
|
UpgradeDiffConfiguration
Appears in:
UpgradeDiffConfiguration contains a list of configurable options which are specific to the kubeadm upgrade diff
command.
Field | Description |
---|---|
kubernetesVersion string |
|
contextLines int |
|
UpgradeNodeConfiguration
Appears in:
UpgradeNodeConfiguration contains a list of configurable options which are specific to the “kubeadm upgrade node” command.
Field | Description |
---|---|
certificateRenewal bool |
|
dryRun bool |
|
etcdUpgrade bool |
|
ignorePreflightErrors []string |
|
skipPhases []string |
|
patches Patches |
|
imagePullPolicy core/v1.PullPolicy |
|
imagePullSerial bool |
|
UpgradePlanConfiguration
Appears in:
UpgradePlanConfiguration contains a list of configurable options which are specific to the “kubeadm upgrade plan” command.
Field | Description |
---|---|
kubernetesVersion [Required]string |
|
allowExperimentalUpgrades bool |
|
allowRCUpgrades bool | Enable |
dryRun bool |
|
ignorePreflightErrors []string |
|
printConfig bool |
|