Upgrade from v1.3.2 to v1.4.0

General information

An Upgrade button appears on the Dashboard screen whenever a new Harvester version that you can upgrade to becomes available. For more information, see Start an upgrade.

For air-gapped environments, see Prepare an air-gapped upgrade.

Known issues

1. A VM with a container disk can’t be migrated which makes the upgrade stuck in pre-drain status

tip

Manually stop the VMs to continue the upgrade process.

When upgrading from v1.3.2 to v1.4.0, the upgrade process may become stuck if a VM with a container disk cannot be migrated. There is some limitation of live migration.

For more information, see Issue #7005.

2. Upgrade stuck on waiting harvester bundle

When upgrading from v1.3.2 to v1.4.0, the upgrade process may become stuck on waiting for the Harvester bundle to become ready. This issue is caused by a race condition when the Fleet agent (fleet-agent) is redeployed.

The following error messages indicate that the issue exists.

> kubectl get bundles -n fleet-local
NAME                                          BUNDLEDEPLOYMENTS-READY   STATUS
mcc-harvester                                 0/1                       ErrApplied(1) [Cluster fleet-local/local: encountered 2 deletion errors. First is: admission webhook "validator.harvesterhci.io" denied the request: Internal error occurred: no route match found for DELETE /v1, Kind=Secret harvester-system/sh.helm.release.v1.harvester.v2]
mcc-harvester-crd                             0/1                       ErrApplied(1) [Cluster fleet-local/local: admission webhook "validator.harvesterhci.io" denied the request: Internal error occurred: no route match found for DELETE /v1, Kind=Secret harvester-system/sh.helm.release.v1.harvester-crd.v1]

You can run the following script to fix the issue.

#!/bin/bash
patch_fleet_bundle() {
  local bundleName=$1
  local generation=$(kubectl get -n fleet-local bundle ${bundleName} -o jsonpath='{.spec.forceSyncGeneration}')
  local new_generation=$((generation+1))
  patch_manifest="$(mktemp)"
  cat > "$patch_manifest" <<EOF
{
  "spec": {
    "forceSyncGeneration": $new_generation
  }
}
EOF
  echo "patch bundle to new generation: $new_generation"
  kubectl patch -n fleet-local bundle ${bundleName}  --type=merge --patch-file $patch_manifest
  rm -f $patch_manifest
}
echo "removing harvester validating webhook"
kubectl delete validatingwebhookconfiguration harvester-validator
for bundle in mcc-harvester-crd mcc-harvester
do
  patch_fleet_bundle ${bundle}
done
echo "removing longhorn services"
kubectl delete svc longhorn-engine-manager -n longhorn-system --ignore-not-found=true
kubectl delete svc longhorn-replica-manager -n longhorn-system --ignore-not-found=true