Upgrade Kubernetes on Azure AKS

    AKS provides az aks upgrade for in-places nodes upgrade by node reimaged, but this will cause the original Longhorn disks missing, then there will be no disks allowing replica rebuilding in upgraded nodes anymore.

    We suggest using node-pool replacement to upgrade the agent nodes but use az aks upgrade for control plane nodes to ensure data safety.

    1. In Longhorn, set replica-replenishment-wait-interval to 0.

    2. Upgrade AKS control plane.

      1. AKS_RESOURCE_GROUP=<aks-resource-group>
      2. AKS_CLUSTER_NAME=<aks-cluster-name>
      3. AKS_K8S_VERSION_UPGRADE=<aks-k8s-version>
      5. az aks upgrade \
      6.     --resource-group ${AKS_RESOURCE_GROUP} \
      7.     --name ${AKS_CLUSTER_NAME} \
      8.     --kubernetes-version ${AKS_K8S_VERSION_UPGRADE} \
      9.     --control-plane-only

    3. Add a new node-pool.

      1. AKS_NODEPOOL_NAME_NEW=<new-nodepool-name>
      2. AKS_DISK_SIZE=<disk-size-in-gb>
      3. AKS_NODE_NUM=<number-of-nodes>
      5. az aks nodepool add \
      6.   --resource-group ${AKS_RESOURCE_GROUP} \
      7.   --cluster-name ${AKS_CLUSTER_NAME} \
      8.   --name ${AKS_NODEPOOL_NAME_NEW} \
      9.   --node-count ${AKS_NODE_NUM} \
      10.   --node-osdisk-size ${AKS_DISK_SIZE} \
      11.   --kubernetes-version ${AKS_K8S_VERSION_UPGRADE} \
      12.   --mode System

    4. Using Longhorn UI to disable the disk scheduling and request eviction for nodes in the old node-pool.

    5. Cordon and drain Kubernetes nodes in the old node-pool.

      1. AKS_NODEPOOL_NAME_OLD=<old-nodepool-name>
      3. for n in `kubectl get nodes | grep ${AKS_NODEPOOL_NAME_OLD}- | awk '{print $1}'`; do
      4.   kubectl cordon $n && \
      5.   kubectl drain $n --ignore-daemonsets --delete-emptydir-data
      6. done

    6. Delete old node-pool.

      1. az aks nodepool delete \
      2.   --cluster-name ${AKS_CLUSTER_NAME} \
      3.   --name ${AKS_NODEPOOL_NAME_OLD} \
      4.   --resource-group ${AKS_RESOURCE_GROUP}

    © 2019-2023 Longhorn Authors | Documentation Distributed under CC-BY-4.0

    © 2023 The Linux Foundation. All rights reserved. The Linux Foundation has registered trademarks and uses trademarks. For a list of trademarks of The Linux Foundation, please see our Trademark Usage page.