Manage Node-Group on Azure AKS

See Create and manage multiple node pools for a cluster in Azure Kubernetes Service (AKS) for more information.

Following is an example to replace cluster nodes with a new storage size.

Storage Expansion

AKS does not support additional disk in its template. It is possible for manual disk attachment. Then raw device needs to be mounted either by manually mounting in VM or during launch with CustomScriptExtension that is not supported in AKS.

  1. In Longhorn, set replica-replenishment-wait-interval to 0.

  2. Add a new node-pool. Later Longhorn components will be automatically deployed on the nodes in this pool.

    1. AKS_NODEPOOL_NAME_NEW=<new-nodepool-name>
    2. AKS_RESOURCE_GROUP=<aks-resource-group>
    3. AKS_CLUSTER_NAME=<aks-cluster-name>
    4. AKS_DISK_SIZE_NEW=<new-disk-size-in-gb>
    5. AKS_NODE_NUM=<number-of-nodes>
    6. AKS_K8S_VERSION=<kubernetes-version>
    8. az aks nodepool add \
    9.   --resource-group ${AKS_RESOURCE_GROUP} \
    10.   --cluster-name ${AKS_CLUSTER_NAME} \
    11.   --name ${AKS_NODEPOOL_NAME_NEW} \
    12.   --node-count ${AKS_NODE_NUM} \
    13.   --node-osdisk-size ${AKS_DISK_SIZE_NEW} \
    14.   --kubernetes-version ${AKS_K8S_VERSION} \
    15.   --mode System

  3. Using Longhorn UI to disable the disk scheduling and request eviction for nodes in the old node-pool.

  4. Cordon and drain Kubernetes nodes in the old node-pool.

    1. AKS_NODEPOOL_NAME_OLD=<old-nodepool-name>
    3. for n in `kubectl get nodes | grep ${AKS_NODEPOOL_NAME_OLD}- | awk '{print $1}'`; do
    4.   kubectl cordon $n && \
    5.   kubectl drain $n --ignore-daemonsets --delete-emptydir-data
    6. done

  5. Delete old node-pool.

    1. az aks nodepool delete \
    2.   --cluster-name ${AKS_CLUSTER_NAME} \
    3.   --name ${AKS_NODEPOOL_NAME_OLD} \
    4.   --resource-group ${AKS_RESOURCE_GROUP}