v1.29.X

Upgrade Notice

Before upgrading from earlier releases, be sure to read the Kubernetes Urgent Upgrade Notes.

Release v1.29.11+k3s1

This release updates Kubernetes to v1.29.11, and fixes a number of issues.

For more details on what’s new, see the Kubernetes release notes.

Changes since v1.29.10+k3s1:

• Backport E2E GHA fixes (#11229)
• Backports for 2024-11 (#11263)
• Update flannel and base cni plugins version (#11249)
• Bump to latest k3s-root version in scripts/version.sh (#11300)
• More backports for 2024-11 (#11309)
• Fix issue with loadbalancer failover to default server (#11326)
• Update Kubernetes to v1.29.11-k3s1 (#11370)
• Bump containerd to -k3s2 to fix rewrites (#11405)

Release v1.29.10+k3s1

This release updates Kubernetes to v1.29.10, and fixes a number of issues.

For more details on what’s new, see the Kubernetes release notes.

Changes since v1.29.9+k3s1:

• Add int test for flannel-ipv6masq (#10905)
• Bump Wharfie to v0.6.7 (#10976)
• Add user path to runtimes search (#11004)
• Add e2e test for advanced fields in services (#11021)
• Launch private registry with init (#11046)
• Backports for 2024-10 (#11062)
• Allow additional Rootless CopyUpDirs through K3S_ROOTLESS_COPYUPDIRS (#11043)
• Bump containerd to v1.7.22 (#11074)
• Simplify svclb ds (#11084)
• Add the nvidia runtime cdi (#11094)
• Revert “Make svclb as simple as possible” (#11114)
• Fixes “file exists” error from CNI bins when upgrading k3s (#11127)
• Update to Kubernetes v1.29.10-k3s1 and Go 1.22.8 (#11160)

Release v1.29.9+k3s1

This release updates Kubernetes to v1.29.9, and fixes a number of issues.

For more details on what’s new, see the Kubernetes release notes.

Changes since v1.29.8+k3s1:

• Update CNI plugins version (#10819)
• Backports for 2024-09 (#10844)
• Testing And Secrets-Encryption Backports for 2024-09 (#10803)
  • Update to newer OS images for install testing
  • Fix caching name for e2e vagrant box
  • Fix deploy latest commit on E2E tests
  • Remove secrets encryption controller #10612
  • DRY E2E Upgrade test setup
  • Cover edge case when on new minor release for E2E upgrade test
• Fix hosts.toml header var (#10873)
• Update to v1.29.9-k3s1 and Go 1.22.6 (#10885)
• Update Kubernetes to v1.29.9-k3s2 (#10908)

Release v1.29.8+k3s1

This release updates Kubernetes to v1.29.8, and fixes a number of issues.

For more details on what’s new, see the Kubernetes release notes.

Changes since v1.29.7+k3s1:

• Fixing setproctitle function (#10623)
• Bump docker/docker to v25.0.6 (#10650)
• Backports for 2024-08 release cycle (#10665)
  • Use pagination when listing large numbers of resources
  • Fix multiple issues with servicelb
  • Remove deprecated use of wait. functions
  • Wire lasso metrics up to metrics endpoint
• Backports for August 2024 (#10672)
• Bump containerd to v1.7.20 (#10661)
• Add tolerations support for DaemonSet pods (#10704)
  • New Feature: Users can now define Kubernetes tolerations for ServiceLB DaemonSet directly in the svccontroller.k3s.cattle.io/tolerations annotation on services.
• Update to v1.29.8-k3s1 and Go 1.22.5 (#10720)

Release v1.29.7+k3s1

This release updates Kubernetes to v1.29.7, and fixes a number of issues.

For more details on what’s new, see the Kubernetes release notes.

Changes since v1.29.6+k3s2:

• Backports for 2024-07 release cycle (#10498)
  • Bump k3s-root to v0.14.0
  • Bump github.com/hashicorp/go-retryablehttp from 0.7.4 to 0.7.7
  • Bump Local Path Provisioner version
  • Ensure remotedialer kubelet connections use kubelet bind address
  • Chore: Bump Trivy version
  • Add etcd s3 config secret implementation
• July Test Backports (#10508)
• Update to v1.29.7-k3s1 and Go 1.22.5 (#10539)
• Fix issues loading data-dir value from env vars or dropping config files (#10597)

Release v1.29.6+k3s2

This release updates Kubernetes to v1.29.6, and fixes a number of issues.

For more details on what’s new, see the Kubernetes release notes.

Changes since v1.29.6+k3s1:

• Update flannel to v0.25.4 and fixed issue with IPv6 mask (#10427)

Release v1.29.6+k3s1

This release updates Kubernetes to v1.29.6, and fixes a number of issues.

For more details on what’s new, see the Kubernetes release notes.

Changes since v1.29.5+k3s1:

• Fix bug when using tailscale config by file (#10142)
• Bump flannel version to v0.25.2 (#10220)
• Update kube-router version to v2.1.2 (#10181)
• Improve tailscale test & add extra log in e2e tests (#10212)
• Backports for 2024-06 release cycle (#10249)
  • Add WithSkipMissing to not fail import on missing blobs
  • Use fixed stream server bind address for cri-dockerd
  • Switch stargz over to cri registry config_path
  • Bump to containerd v1.7.17, etcd v3.5.13
  • Bump spegel version
  • Fix issue with externalTrafficPolicy: Local for single-stack services on dual-stack nodes
  • ServiceLB now sets the priorityClassName on svclb pods to system-node-critical by default. This can be overridden on a per-service basis via the svccontroller.k3s.cattle.io/priorityclassname annotation.
  • Bump minio-go to v7.0.70
  • Bump kine to v0.11.9 to fix pagination
  • Update valid resolv conf
  • Add missing kernel config check
  • Symlinked sub-directories are now respected when scanning Auto-Deploying Manifests (AddOns)
  • Fix bug: allow helm controller set owner reference
  • Bump klipper-helm image for tls secret support
  • Fix issue with k3s-etcd informers not starting
  • --Enable-pprof can now be set on agents to enable the debug/pprof endpoints. When set, agents will listen on the supervisor port.
  • --Supervisor-metrics can now be set on servers to enable serving internal metrics on the supervisor endpoint; when set agents will listen on the supervisor port.
  • Fix netpol crash when node remains tainted uninitialized
  • The embedded load-balancer will now fall back to trying all servers with health-checks ignored, if all servers have been marked unavailable due to failed health checks.
• More backports for 2024-06 release cycle (#10288)
• Add snapshot retention etcd-s3-folder fix (#10316)
• Add test for isValidResolvConf (#10302) (#10329)
• Fix race condition panic in loadbalancer.nextServer (#10322)
• Fix typo, use rancher/permissions (#10298)
• Expand GHA go caching to include newest release branch (#10334)
• Update Kubernetes to v1.29.6 (#10348)
• Fix agent supervisor port using apiserver port instead (#10354)
• Fix issue that allowed multiple simultaneous snapshots to be allowed (#10376)

Release v1.29.5+k3s1

This release updates Kubernetes to v1.29.5, and fixes a number of issues.

For more details on what’s new, see the Kubernetes release notes.

Changes since v1.29.4+k3s1:

• Update stable channel to v1.29.4+k3s1 (#10031)
• Add E2E Split Server to Drone, support parallel testing in Drone (#9940)
• Bump E2E opensuse leap to 15.6, fix btrfs test (#10057)
• Replace deprecated ruby function (#10091)
• Set correct release channel for e2e upgrade test (#10106)
• Windows changes (#10115)
• Update to v1.29.5-k3s1 and Go 1.21.9 (#10108)

Release v1.29.4+k3s1

This release updates Kubernetes to v1.29.4, and fixes a number of issues.

For more details on what’s new, see the Kubernetes release notes.

Changes since v1.29.3+k3s1:

• Send error response if member list cannot be retrieved (#9722)
• Respect cloud-provider fields set by kubelet (#9721)
  • The k3s stub cloud provider now respects the kubelet’s requested provider-id, instance type, and topology labels
• Fix error when image has already been pulled (#9770)
• Add a new error when kine is with disable apiserver or disable etcd (#9766)
• Bump k3s-root to v0.13.0 (#9718)
• Use ubuntu latest for better golang caching keys (#9711)
• Bump Trivy version (#9780)
• Move to ubuntu 23.10 for E2E tests (#9755)
• Update channel server (#9808)
• Add /etc/passwd and /etc/group to k3s docker image (#9784)
• Fix etcd snapshot reconcile for agentless servers (#9809)
• Add health-check support to loadbalancer (#9757)
• Add tls for kine (#9572)
  • Kine is now able to use TLS
• Transition from deprecated pointer library to ptr (#9801)
• Remove old pinned dependencies (#9806)
• Several E2E Matrix improvements (#9802)
• Add certificate expiry check, events, and metrics (#9772)
• Add updatecli policy to update k3s-root (#9844)
• Bump Trivy version (#9840)
• Add workaround for containerd hosts.toml bug when passing config for default registry endpoint (#9853)
• Fix: agent volume in example docker compose (#9838)
• Bump spegel to v0.0.20-k3s1 (#9863)
• Add supervisor cert/key to rotate list (#9832)
• Add quotes to avoid useless updatecli updates (#9877)
• Bump containerd and cri-dockerd (#9886)
  • The embedded containerd has been bumped to v1.7.15
  • The embedded cri-dockerd has been bumped to v0.3.12
• Move etcd snapshot management CLI to request/response (#9816)
  • The k3s etcd-snapshot command has been reworked for improved consistency. All snapshots operations are now performed by the server process, with the CLI acting as a client to initiate and report results. As a side effect, the CLI is now less noisy when managing snapshots.
• Improve etcd load-balancer startup behavior (#9883)
• Actually fix agent certificate rotation (#9902)
• Bump latest to v1.29.3+k3s1 (#9909)
• Update packaged manifests (#9920)
  • Traefik has been bumped to v2.10.7.
  • Traefik pod annotations are now set properly in the default chart values.
  • The system-default-registry value now supports RFC2732 IPv6 literals.
  • The local-path provisioner now defaults to creating local volumes, instead of hostPath.
• Allow Local path provisioner to read helper logs (#9835)
• Update kube-router to v2.1.0 (#9926)
• Match setup-go caching key in GitHub Actions (#9890)
• Add startup testlet on preloaded images (#9941)
• Update to v1.29.4-k3s1 and Go 1.21.9 (#9960)
• Fix on-demand snapshots timing out; not honoring folder (#9984)
• Make /db/info available anonymously from localhost (#10001)

Release v1.29.3+k3s1

This release updates Kubernetes to v1.29.3, and fixes a number of issues.

For more details on what’s new, see the Kubernetes release notes.

Changes since v1.29.2+k3s1:

• Testing ADR (#9562)
• Unit Testing Matrix and Actions bump (#9479)
• Update install test OS matrix (#9480)
• Update klipper-lb image version (#9488)
• Add an integration test for flannel-backend=none (#9582)
• Better GitHub CI caching strategy for golang (#9495)
• Correct formatting of GH PR sha256sum artifact (#9472)
• Rootless mode also bind service nodePort to host for LoadBalancer type (#9512)
  • Rootless mode should also bind service nodePort to host for LoadBalancer type, matching UX of rootful mode.
• Fix coredns NodeHosts on dual-stack clusters (#9584)
• Tweak netpol node wait logs (#9581)
• Fix issue with etcd node name missing hostname (#9522)
• Bump helm-controller/klipper-helm versions (#9595)
• Update stable channel to v1.28.7+k3s1 (#9615)
• Reenable Install and Snapshotter Testing (#9601)
• Move docker tests into tests folder (#9555)
• Fix setup-go typo (#9634)
• Fix additional corner cases in registries handling (#9556)
• Fix snapshot prune (#9502)
• Use and version flannel/cni-plugin properly (#9635)
  • The embedded flannel cni-plugin binary is now built and versioned separate from the rest of the cni plugins and the embedded flannel controller.
• Bump spegel (#9599)
  • Bump spegel to v0.0.18-k3s3
  • Adds wildcard registry support
  • Fixes issue with excessive CPU utilization while waiting for containerd to start
  • Add env var to allow spegel mirroring of latest tag
• Chore(deps): Remediating CVEs found by trivy; CVE-2023-45142 on otelrestful and CVE-2023-48795 on golang.org/x/crypto (#9513)
• Fix: use correct wasm shims names (#9519)
• Fix wildcard with embedded registry test (#9649)
• Disable color outputs using NO_COLOR env var (#9357)
  • To enable raw output for the check-config subcommand, you may now set NO_COLOR=1
• Improve tailscale e2e test (#9586)
• Adjust first node-ip based on configured clusterCIDR (#9520)
• Bump Trivy version (#9528)
• Include flannel version in flannel cni plugin version (#9648)
  • The flannel controller version is now reported as build metadata on the flannel cni plugin version.
• Enable E2E tests on GitHub Actions (#9660)
• Bump metrics-server to v0.7.0 (#9673)
• Bump upload and download actions to v4 (#9666)
• Warn and suppress duplicate registry mirror endpoints (#9697)
  • K3s will now warn and suppress duplicate entries in the mirror endpoint list for a registry. Containerd does not support listing the same endpoint multiple times as a mirror for a single upstream registry.
• Remove repetitive words (#9671)
• Run Subset of Docker tests in GitHub Actions (#9698)
• Fix wildcard entry upstream fallback (#9729)
• Update to v1.29.3-k3s1 and Go 1.21.8 (#9747)

Release v1.29.2+k3s1

This release updates Kubernetes to v1.29.2, and fixes a number of issues.

For more details on what’s new, see the Kubernetes release notes.

Changes since v1.29.1+k3s2:

• Bump Local Path Provisioner version (#8953)
• Add ability to install K3s PR Artifact from GitHub (#9185)
  • Adds INSTALL_K3S_PR option to install a build of K3s from any open PR with CI approval
• Bump Trivy version (#9237)
• Bump codecov/codecov-action from 3 to 4 (#9353)
• Update stable channel (#9388)
• Fix snapshot reconcile retry (#9318)
• Add check for etcd-snapshot-dir and fix panic in Walk (#9317)
• Bump CNI plugins to v1.4.0 (#9249)
• Fix issue with coredns node hosts controller (#9354)
  • Fixed issue that could cause coredns pods to fail to start when the embedded helm controller is disabled, due to the configmap not being updated with node hosts entries.
• Fix on-demand snapshots on ipv6-only nodes (#9247)
• Bump flannel version (#9395)
  • Bumped flannel to v0.24.2
• Build: Align drone base images (#8959)
• Changed how lastHeartBeatTime works in the etcd condition (#9263)
• Runtimes refactor using exec.LookPath (#9311)
  • Directories containing runtimes need to be included in the $PATH environment variable for effective runtime detection.
• Bump cri-dockerd to fix compat with Docker Engine 25 (#9290)
• Add codcov secret for integration tests on Push (#9422)
• Allow executors to define containerd and cridockerd behavior (#9184)
• Update Kube-router to v2.0.1 (#9396)
• : Test_UnitApplyContainerdQoSClassConfigFileIfPresent (Created) (#8945)
• Readd k3s secrets-encrypt rotate-keys with correct support for KMSv2 GA (#9340)
• Fix iptables check when sbin isn’t in user PATH (#9344)
• Don’t create NodePasswordValidationFailed event if agent is disabled (#9312)
  • The NodePasswordValidationFailed Events will no longer be emitted, if the agent is disabled.
• Expose rootless state dir under ~/.rancher/k3s/rootless (#9308)
  • When running k3s in rootless mode, expose rootlesskit’s state directory as ~/.rancher/k3s/rootless
• Expose rootless containerd socket directories for external access (#9309)
  • Mount k3s rootless containerd & cri-dockerd socket directories to $XDG_RUNTIME_DIR/k3s/containerd and $XDG_RUNTIME_DIR/k3s/cri-dockerd respectively.
• Bump kine and set NotifyInterval to what the apiserver expects (#9349)
• Update Kubernetes to v1.29.2 (#9493)
• Fix drone publish for arm (#9503)
• Remove failing Drone step (#9517)
• Restore original order of agent startup functions (#9539)
• Fix netpol startup when flannel is disabled (#9571)

Release v1.29.1+k3s2

This release updates Kubernetes to v1.29.1, and fixes a number of issues.

For more details on what’s new, see the Kubernetes release notes.

Important Notes

Addresses the runc CVE: CVE-2024-21626 by updating runc to v1.1.12.

Changes since v1.29.0+k3s1:

• Bump Sonobuoy version (#8910)
• Bump actions/setup-go from 4 to 5 (#9036)
• Chore: Update Code of Conduct to Redirect to CNCF CoC (#9104)
  • NONE
• Update stable channel to v1.28.5+k3s1 and add v1.29 channel (#9110)
• Added support for env *_PROXY variables for agent loadbalancer (#9070)
  • HTTP_PROXY, HTTPS_PROXY and NO_PROXY environment variables are now taken into account by the agent loadbalancer if K3S_AGENT_HTTP_PROXY_ALLOWED env variable is set to true.
  • This however doesn’t affect local requests as the function used prevents that: https://pkg.go.dev/net/http#ProxyFromEnvironment.
• Add a retry around updating a secrets-encrypt node annotations (#9039)
• Silence SELinux warning on INSTALL_K3S_SKIP_SELINUX_RPM (#8703)
• Add ServiceLB support for PodHostIPs FeatureGate (#8917)
• Added support for env *_PROXY variables for agent loadbalancer (#9118)
• Redirect error stream to null when checking nm-cloud systemd unit (#8815)
  • Remove confusing “nm-cloud-setup.service: No such file or directory” journalctl log
• Dockerfile.dapper: set $HOME properly (#9090)
• Add system-agent-installer-k3s step to GA release instructions (#9153)
• Fix install script checksum (#9159)
• Fix the OTHER etcd snapshot s3 log message that prints the wrong variable (#8944)
• Handle logging flags when parsing kube-proxy args (#8916)
• Fix nil map in full snapshot configmap reconcile (#9049)
• Add support for containerd cri registry config_path (#8973)
• Add more paths to crun runtime detection (#9086)
• Add runtime checking of golang version (#9054)
• Fix OS PRETTY_NAME on tagged releases (#9062)
• Print error when downloading file error inside install script (#6874)
• Wait for cloud-provider taint to be gone before starting the netpol controller (#9076)
• Bump Trivy version (#8812)
• Use ipFamilyPolicy: RequireDualStack for dual-stack kube-dns (#8984)
• Handle etcd status condition when node is not ready and disable etcd (#9084)
• Update s3 e2e test (#9025)
• Add e2e startup test for rootless k3s (#8383)
• Add spegel distributed registry mirror (#8977)
• Bump quic-go for CVE-2023-49295 (#9208)
• Enable network policy controller metrics (#9195)
  • Kube-router network policy controller metrics are now exposed via the default node metrics endpoint
• Fix nonexistent dependency repositories (#9213)
• Move proxy dialer out of init() and fix crash when using K3S_AGENT_HTTP_PROXY_ALLOWED=true (#9219)
• Error getting node in setEtcdStatusCondition (#9210)
• Update to v1.29.1 and Go 1.21.6 (#9259)
• New stale action (#9278)
• Fix handling of bare hostname or IP as endpoint address in registries.yaml (#9323)
• Bump runc to v1.1.12 and helm-controller to v0.15.7 (#9332)
• Bump helm-controller to fix issue with ChartContent (#9345)

Release v1.29.0+k3s1

This release is K3S’s first in the v1.29 line. This release updates Kubernetes to v1.29.0.

Before upgrading from earlier releases, be sure to read the Kubernetes Urgent Upgrade Notes.

Important

This release removes the experimental rotate-keys subcommand due to changes in Kubernetes upstream for KMSv2, the subcommand should be added back in future releases.

Important

This release also removes the multi-cluster-cidr flag, since the support for this alpha feature has been removed completely from Kubernetes upstream, this flag should be removed from the configuration before upgrade.

Changes since v1.28.4+k3s2:

• Fix overlapping address range (#8913)
• Modify CONTRIBUTING.md guide (#8954)
• Nov 2023 stable channel update (#9022)
• Default runtime and runtime classes for wasm/nvidia/crun (#8936)
  • Added runtime classes for wasm/nvidia/crun
  • Added default runtime flag for containerd
• Bump containerd/runc to v1.7.10-k3s1/v1.1.10 (#8962)
• Allow setting default-runtime on servers (#9027)
• Bump containerd to v1.7.11 (#9040)
• Remove GA feature-gates (#8970)
• Only publish to code_cov on merged E2E builds (#9051)
• Update Kubernetes to v1.29.0+k3s1 (#9052)
• Update flannel to v0.24.0 and remove multiclustercidr flag (#9075)
• Remove rotate-keys subcommand (#9079)