k3s server

In this section, you’ll learn how to configure the K3s server.

Note that servers also run an agent, so all of the configuration options listed in the k3s agent documentation are also supported on servers.

Options are documented on this page as CLI flags, but can also be passed as configuration file options. See the Configuration File documentation for more information on using YAML configuration files.

Critical Configuration Values

The following options must be set to the same value on all servers in the cluster. Failure to do so will cause new servers to fail to join the cluster when using embedded etcd, or incorrect operation of the cluster when using an external datastore.

  • --agent-token
  • --cluster-cidr
  • --cluster-dns
  • --cluster-domain
  • --disable-cloud-controller
  • --disable-helm-controller
  • --disable-network-policy
  • --disable=servicelb note: other packaged components may be disabled on a per-server basis
  • --egress-selector-mode
  • --embedded-registry
  • --flannel-backend
  • --flannel-external-ip
  • --flannel-ipv6-masq
  • --secrets-encryption
  • --service-cidr

Commonly Used Options

Database

FlagEnvironment VariableDefaultDescription
—datastore-endpoint valueK3SDATASTORE_ENDPOINTSpecify etcd, NATS, MySQL, Postgres, or SQLite data source name
—datastore-cafile valueK3S_DATASTORE_CAFILETLS Certificate Authority file used to secure datastore backend communication
—datastore-certfile valueK3S_DATASTORE_CERTFILETLS certification file used to secure datastore backend communication
—datastore-keyfile valueK3S_DATASTORE_KEYFILETLS key file used to secure datastore backend communication
—etcd-expose-metricsfalseExpose etcd metrics to client interface
—etcd-disable-snapshotsfalseDisable automatic etcd snapshots
—etcd-snapshot-name value“etcd-snapshot-<unix-timestamp>”Set the base name of etcd snapshots.
—etcd-snapshot-schedule-cron value“0 /12 Snapshot interval time in cron spec. eg. every 5 hours ‘0 */5 * _’
—etcd-snapshot-retention value5Number of snapshots to retain
—etcd-snapshot-dir value${data-dir}/db/snapshotsDirectory to save db snapshots
—etcd-s3Enable backup to S3
—etcd-s3-endpoint value“s3.amazonaws.com”S3 endpoint url
—etcd-s3-endpoint-ca valueS3 custom CA cert to connect to S3 endpoint
—etcd-s3-skip-ssl-verifyDisables S3 SSL certificate validation
—etcd-s3-access-key valueAWS_ACCESS_KEY_IDS3 access key
—etcd-s3-secret-key valueAWS_SECRET_ACCESS_KEYS3 secret key
—etcd-s3-bucket valueS3 bucket name
—etcd-s3-region value“us-east-1”S3 region / bucket location (optional)
—etcd-s3-folder valueS3 folder
—etcd-s3-proxyProxy server to use when connecting to S3, overriding any proxy-releated environment variables
—etcd-s3-config-secretName of secret in the kube-system namespace used to configure S3, if etcd-s3 is enabled and no other etcd-s3 options are set
—etcd-s3-insecureDisables S3 over HTTPS
—etcd-s3-timeout value5m0sS3 timeout (default: 5m0s)

Cluster Options

FlagEnvironment VariableDescription
—token value, -t valueK3S_TOKENShared secret used to join a server or agent to a cluster
—token-file valueK3S_TOKEN_FILEFile containing the cluster-secret/token
—agent-token valueK3S_AGENT_TOKENShared secret used to join agents to the cluster, but not servers
—agent-token-file valueK3S_AGENT_TOKEN_FILEFile containing the agent secret
—server valueK3S_URLServer to connect to, used to join a cluster
—cluster-initK3S_CLUSTER_INITInitialize a new cluster using embedded Etcd
—cluster-resetK3S_CLUSTER_RESETForget all peers and become sole member of a new cluster

Admin Kubeconfig Options

FlagEnvironment VariableDescription
—write-kubeconfig value, -o valueK3S_KUBECONFIG_OUTPUTWrite kubeconfig for admin client to this file
—write-kubeconfig-mode valueK3S_KUBECONFIG_MODEWrite kubeconfig with this mode. The kubeconfig file is owned by root, and written with a default mode of 600. Changing the mode to 644 will allow it to be read by other unprivileged users on the host.
—write-kubeconfig-group valueK3S_KUBECONFIG_GROUPWrite kubeconfig group. Combining with —write-kubeconfig-mode, it will allow your k3s administrators accessing the kubeconfig file but keeping the file owned by root.

Advanced Options

Logging

FlagDefaultDescription
—debugN/ATurn on debug logs
-v value0Number for the log level verbosity
—vmodule valueN/AComma-separated list of FILE_PATTERN=LOG_LEVEL settings for file-filtered logging
—log value, -l valueN/ALog to file
—alsologtostderrN/ALog to standard error as well as file (if set)

Listeners

FlagDefaultDescription
—bind-address value0.0.0.0k3s bind address
—https-listen-port value6443HTTPS listen port
—advertise-address valuenode-external-ip/node-ipIPv4/IPv6 address that apiserver advertises for its service endpoint
Note that the primary service-cidr IP range must be of the same address family as the advertised address
—advertise-port valuelisten-port/0Port that apiserver uses to advertise to members of the cluster
—tls-san valueN/AAdd additional hostnames or IPv4/IPv6 addresses as Subject Alternative Names on the TLS cert
—tls-san-securitytrueProtect the server TLS cert by refusing to add Subject Alternative Names not associated with the kubernetes apiserver service, server nodes, or values of the tls-san option

Data

FlagDefaultDescription
—data-dir value, -d value/var/lib/rancher/k3s or ${HOME}/.rancher/k3s if not rootFolder to hold state

Secrets Encryption

FlagDefaultDescription
—secrets-encryptionfalseEnable Secret encryption at rest

Networking

FlagDefaultDescription
—cluster-cidr value“10.42.0.0/16”IPv4/IPv6 network CIDRs to use for pod IPs
—service-cidr value“10.43.0.0/16”IPv4/IPv6 network CIDRs to use for service
—service-node-port-range value“30000-32767”Port range to reserve for services with NodePort visibility
—cluster-dns value“10.43.0.10”IPv4 Cluster IP for coredns service. Should be in your service-cidr range
—cluster-domain value“cluster.local”Cluster Domain
—flannel-backend value“vxlan”One of ‘none’, ‘vxlan’, ‘ipsec’(deprecated), ‘host-gw’, ‘wireguard-native’, or ‘wireguard’(deprecated)
—flannel-ipv6-masq“N/A”Enable IPv6 masquerading for pod
—flannel-external-ip“N/A”Use node external IP addresses for Flannel traffic
—servicelb-namespace value“kube-system”Namespace of the pods for the servicelb component
—egress-selector-mode value“agent”Must be one of the following:
  • disabled: The apiserver does not use agent tunnels to communicate with nodes. Requires that servers run agents, and have direct connectivity to the kubelet on agents, or the apiserver will not be able to function access service endpoints or perform kubectl exec and kubectl logs.
  • agent: The apiserver uses agent tunnels to communicate with nodes. Nodes allow the tunnel connection from loopback addresses. Requires that servers also run agents, or the apiserver will not be able to access service endpoints. The historical default for k3s.
  • pod: The apiserver uses agent tunnels to communicate with nodes and service endpoints, routing endpoint connections to the correct agent by watching Nodes. Nodes allow the tunnel connection from loopback addresses, or a CIDR assigned to their node.
  • cluster: The apiserver uses agent tunnels to communicate with nodes and service endpoints, routing endpoint connections to the correct agent by watching Endpoints. Nodes allow the tunnel connection from loopback addresses, or the configured cluster CIDR range.

Storage Class

FlagDescription
—default-local-storage-path valueDefault local storage path for local provisioner storage class

Kubernetes Components

FlagDescription
—disable valueSee “Using the —disable flag
—disable-schedulerDisable Kubernetes default scheduler
—disable-cloud-controllerDisable k3s default cloud controller manager
—disable-kube-proxyDisable running kube-proxy
—disable-network-policyDisable k3s default network policy controller
—disable-helm-controllerDisable Helm controller

Customized Flags for Kubernetes Processes

FlagDescription
—etcd-arg valueCustomized flag for etcd process
—kube-apiserver-arg valueCustomized flag for kube-apiserver process
—kube-scheduler-arg valueCustomized flag for kube-scheduler process
—kube-controller-manager-arg valueCustomized flag for kube-controller-manager process
—kube-cloud-controller-manager-arg valueCustomized flag for kube-cloud-controller-manager process
—kubelet-arg valueCustomized flag for kubelet process
—kube-proxy-arg valueCustomized flag for kube-proxy process

Experimental Options

FlagDescription
—rootlessRun rootless
—enable-pprofEnable pprof endpoint on supervisor port
—dockerUse cri-dockerd instead of containerd
—prefer-bundled-binPrefer bundled userspace binaries over host binaries
—disable-agentSee “Running Agentless Servers
—embedded-registrySee “Embedded Registry Mirror
—vpn-authSee “Integration with the Tailscale VPN provider
—vpn-auth-fileSee “Integration with the Tailscale VPN provider

Deprecated Options

FlagEnvironment VariableDescription
—no-flannelN/AUse —flannel-backend=none
—no-deploy valueN/AUse —disable
—cluster-secret valueK3S_CLUSTER_SECRETUse —token
—flannel-backend wireguardN/AUse —flannel-backend=wireguard-native
—flannel-backend value=option1=valueN/AUse —flannel-conf to specify the flannel config file with the backend config

K3s Server CLI Help

If an option appears in brackets below, for example [$K3S_TOKEN], it means that the option can be passed in as an environment variable of that name.

  1. NAME:
  2. k3s server - Run management server
  3. USAGE:
  4. k3s server [OPTIONS]
  5. OPTIONS:
  6. --config FILE, -c FILE (config) Load configuration from FILE (default: "/etc/rancher/k3s/config.yaml") [$K3S_CONFIG_FILE]
  7. --debug (logging) Turn on debug logs [$K3S_DEBUG]
  8. -v value (logging) Number for the log level verbosity (default: 0)
  9. --vmodule value (logging) Comma-separated list of FILE_PATTERN=LOG_LEVEL settings for file-filtered logging
  10. --log value, -l value (logging) Log to file
  11. --alsologtostderr (logging) Log to standard error as well as file (if set)
  12. --bind-address value (listener) k3s bind address (default: 0.0.0.0)
  13. --https-listen-port value (listener) HTTPS listen port (default: 6443)
  14. --advertise-address value (listener) IPv4/IPv6 address that apiserver uses to advertise to members of the cluster (default: node-external-ip/node-ip)
  15. --advertise-port value (listener) Port that apiserver uses to advertise to members of the cluster (default: listen-port) (default: 0)
  16. --tls-san value (listener) Add additional hostnames or IPv4/IPv6 addresses as Subject Alternative Names on the server TLS cert
  17. --tls-san-security (listener) Protect the server TLS cert by refusing to add Subject Alternative Names not associated with the kubernetes apiserver service, server nodes, or values of the tls-san option (default: true)
  18. --data-dir value, -d value (data) Folder to hold state default /var/lib/rancher/k3s or ${HOME}/.rancher/k3s if not root [$K3S_DATA_DIR]
  19. --cluster-cidr value (networking) IPv4/IPv6 network CIDRs to use for pod IPs (default: 10.42.0.0/16)
  20. --service-cidr value (networking) IPv4/IPv6 network CIDRs to use for service IPs (default: 10.43.0.0/16)
  21. --service-node-port-range value (networking) Port range to reserve for services with NodePort visibility (default: "30000-32767")
  22. --cluster-dns value (networking) IPv4 Cluster IP for coredns service. Should be in your service-cidr range (default: 10.43.0.10)
  23. --cluster-domain value (networking) Cluster Domain (default: "cluster.local")
  24. --flannel-backend value (networking) Backend (valid values: 'none', 'vxlan', 'host-gw', 'wireguard-native' (default: "vxlan")
  25. --flannel-ipv6-masq (networking) Enable IPv6 masquerading for pod
  26. --flannel-external-ip (networking) Use node external IP addresses for Flannel traffic
  27. --egress-selector-mode value (networking) One of 'agent', 'cluster', 'pod', 'disabled' (default: "agent")
  28. --servicelb-namespace value (networking) Namespace of the pods for the servicelb component (default: "kube-system")
  29. --write-kubeconfig value, -o value (client) Write kubeconfig for admin client to this file [$K3S_KUBECONFIG_OUTPUT]
  30. --write-kubeconfig-mode value (client) Write kubeconfig with this mode [$K3S_KUBECONFIG_MODE]
  31. --write-kubeconfig-group value (client) Write kubeconfig with this group [$K3S_KUBECONFIG_GROUP]
  32. --helm-job-image value (helm) Default image to use for helm jobs
  33. --token value, -t value (cluster) Shared secret used to join a server or agent to a cluster [$K3S_TOKEN]
  34. --token-file value (cluster) File containing the token [$K3S_TOKEN_FILE]
  35. --agent-token value (cluster) Shared secret used to join agents to the cluster, but not servers [$K3S_AGENT_TOKEN]
  36. --agent-token-file value (cluster) File containing the agent secret [$K3S_AGENT_TOKEN_FILE]
  37. --server value, -s value (cluster) Server to connect to, used to join a cluster [$K3S_URL]
  38. --cluster-init (cluster) Initialize a new cluster using embedded Etcd [$K3S_CLUSTER_INIT]
  39. --cluster-reset (cluster) Forget all peers and become sole member of a new cluster [$K3S_CLUSTER_RESET]
  40. --cluster-reset-restore-path value (db) Path to snapshot file to be restored
  41. --kube-apiserver-arg value (flags) Customized flag for kube-apiserver process
  42. --etcd-arg value (flags) Customized flag for etcd process
  43. --kube-controller-manager-arg value (flags) Customized flag for kube-controller-manager process
  44. --kube-scheduler-arg value (flags) Customized flag for kube-scheduler process
  45. --kube-cloud-controller-manager-arg value (flags) Customized flag for kube-cloud-controller-manager process
  46. --datastore-endpoint value (db) Specify etcd, NATS, MySQL, Postgres, or SQLite (default) data source name [$K3S_DATASTORE_ENDPOINT]
  47. --datastore-cafile value (db) TLS Certificate Authority file used to secure datastore backend communication [$K3S_DATASTORE_CAFILE]
  48. --datastore-certfile value (db) TLS certification file used to secure datastore backend communication [$K3S_DATASTORE_CERTFILE]
  49. --datastore-keyfile value (db) TLS key file used to secure datastore backend communication [$K3S_DATASTORE_KEYFILE]
  50. --etcd-expose-metrics (db) Expose etcd metrics to client interface. (default: false)
  51. --etcd-disable-snapshots (db) Disable automatic etcd snapshots
  52. --etcd-snapshot-name value (db) Set the base name of etcd snapshots (default: etcd-snapshot-<unix-timestamp>) (default: "etcd-snapshot")
  53. --etcd-snapshot-schedule-cron value (db) Snapshot interval time in cron spec. eg. every 5 hours '0 */5 * * *' (default: "0 */12 * * *")
  54. --etcd-snapshot-retention value (db) Number of snapshots to retain (default: 5)
  55. --etcd-snapshot-dir value (db) Directory to save db snapshots. (default: ${data-dir}/db/snapshots)
  56. --etcd-snapshot-compress (db) Compress etcd snapshot
  57. --etcd-s3 (db) Enable backup to S3
  58. --etcd-s3-endpoint value (db) S3 endpoint url (default: "s3.amazonaws.com")
  59. --etcd-s3-endpoint-ca value (db) S3 custom CA cert to connect to S3 endpoint
  60. --etcd-s3-skip-ssl-verify (db) Disables S3 SSL certificate validation
  61. --etcd-s3-access-key value (db) S3 access key [$AWS_ACCESS_KEY_ID]
  62. --etcd-s3-secret-key value (db) S3 secret key [$AWS_SECRET_ACCESS_KEY]
  63. --etcd-s3-bucket value (db) S3 bucket name
  64. --etcd-s3-region value (db) S3 region / bucket location (optional) (default: "us-east-1")
  65. --etcd-s3-folder value (db) S3 folder
  66. --etcd-s3-proxy value (db) Proxy server to use when connecting to S3, overriding any proxy-releated environment variables
  67. --etcd-s3-config-secret value (db) Name of secret in the kube-system namespace used to configure S3, if etcd-s3 is enabled and no other etcd-s3 options are set
  68. --etcd-s3-insecure (db) Disables S3 over HTTPS
  69. --etcd-s3-timeout value (db) S3 timeout (default: 5m0s)
  70. --default-local-storage-path value (storage) Default local storage path for local provisioner storage class
  71. --disable value (components) Do not deploy packaged components and delete any deployed components (valid items: coredns, servicelb, traefik, local-storage, metrics-server, runtimes)
  72. --disable-scheduler (components) Disable Kubernetes default scheduler
  73. --disable-cloud-controller (components) Disable k3s default cloud controller manager
  74. --disable-kube-proxy (components) Disable running kube-proxy
  75. --disable-network-policy (components) Disable k3s default network policy controller
  76. --disable-helm-controller (components) Disable Helm controller
  77. --embedded-registry (experimental/components) Enable embedded distributed container registry; requires use of embedded containerd; when enabled agents will also listen on the supervisor port
  78. --supervisor-metrics (experimental/components) Enable serving k3s internal metrics on the supervisor port; when enabled agents will also listen on the supervisor port
  79. --node-name value (agent/node) Node name [$K3S_NODE_NAME]
  80. --with-node-id (agent/node) Append id to node name
  81. --node-label value (agent/node) Registering and starting kubelet with set of labels
  82. --node-taint value (agent/node) Registering kubelet with set of taints
  83. --image-credential-provider-bin-dir value (agent/node) The path to the directory where credential provider plugin binaries are located (default: "/var/lib/rancher/credentialprovider/bin")
  84. --image-credential-provider-config value (agent/node) The path to the credential provider plugin config file (default: "/var/lib/rancher/credentialprovider/config.yaml")
  85. --docker (agent/runtime) (experimental) Use cri-dockerd instead of containerd
  86. --container-runtime-endpoint value (agent/runtime) Disable embedded containerd and use the CRI socket at the given path; when used with --docker this sets the docker socket path
  87. --default-runtime value (agent/runtime) Set the default runtime in containerd
  88. --image-service-endpoint value (agent/runtime) Disable embedded containerd image service and use remote image service socket at the given path. If not specified, defaults to --container-runtime-endpoint.
  89. --disable-default-registry-endpoint (agent/containerd) Disables containerd fallback default registry endpoint when a mirror is configured for that registry
  90. --nonroot-devices (agent/containerd) Allows non-root pods to access devices by setting device_ownership_from_security_context=true in the containerd CRI config
  91. --pause-image value (agent/runtime) Customized pause image for containerd or docker sandbox (default: "rancher/mirrored-pause:3.6")
  92. --snapshotter value (agent/runtime) Override default containerd snapshotter (default: "overlayfs")
  93. --private-registry value (agent/runtime) Private registry configuration file (default: "/etc/rancher/k3s/registries.yaml")
  94. --system-default-registry value (agent/runtime) Private registry to be used for all system images [$K3S_SYSTEM_DEFAULT_REGISTRY]
  95. --node-ip value, -i value (agent/networking) IPv4/IPv6 addresses to advertise for node
  96. --node-external-ip value (agent/networking) IPv4/IPv6 external IP addresses to advertise for node
  97. --node-internal-dns value (agent/networking) internal DNS addresses to advertise for node
  98. --node-external-dns value (agent/networking) external DNS addresses to advertise for node
  99. --resolv-conf value (agent/networking) Kubelet resolv.conf file [$K3S_RESOLV_CONF]
  100. --flannel-iface value (agent/networking) Override default flannel interface
  101. --flannel-conf value (agent/networking) Override default flannel config file
  102. --flannel-cni-conf value (agent/networking) Override default flannel cni config file
  103. --vpn-auth value (agent/networking) (experimental) Credentials for the VPN provider. It must include the provider name and join key in the format name=<vpn-provider>,joinKey=<key>[,controlServerURL=<url>][,extraArgs=<args>] [$K3S_VPN_AUTH]
  104. --vpn-auth-file value (agent/networking) (experimental) File containing credentials for the VPN provider. It must include the provider name and join key in the format name=<vpn-provider>,joinKey=<key>[,controlServerURL=<url>][,extraArgs=<args>] [$K3S_VPN_AUTH_FILE]
  105. --kubelet-arg value (agent/flags) Customized flag for kubelet process
  106. --kube-proxy-arg value (agent/flags) Customized flag for kube-proxy process
  107. --protect-kernel-defaults (agent/node) Kernel tuning behavior. If set, error if kernel tunables are different than kubelet defaults.
  108. --secrets-encryption Enable secret encryption at rest
  109. --enable-pprof (experimental) Enable pprof endpoint on supervisor port
  110. --rootless (experimental) Run rootless
  111. --prefer-bundled-bin (experimental) Prefer bundled userspace binaries over host binaries
  112. --selinux (agent/node) Enable SELinux in containerd [$K3S_SELINUX]
  113. --lb-server-port value (agent/node) Local port for supervisor client load-balancer. If the supervisor and apiserver are not colocated an additional port 1 less than this port will also be used for the apiserver client load-balancer. (default: 6444) [$K3S_LB_SERVER_PORT]