EdgeFS Upgrades

This guide will walk you through the manual steps to upgrade the software in a Rook EdgeFS cluster from one version to the next. Rook EdgeFS is a multi-cloud distributed software system and therefore there are multiple components to individually upgrade in the sequence defined in this guide. After each component is upgraded, it is important to verify that the cluster returns to a healthy and fully functional state.

We welcome feedback and opening issues!

Supported Versions

The supported version for this upgrade guide is from a 1.0 release to a 1.x releases. Build-to-build upgrades are not guaranteed to work. This guide is to perform upgrades only between the official releases.

Upgrades from Alpha to Beta not supported. However, please see migration procedure below.

EdgeFS Migration

EdgeFS Operator provides a way of preserving data on disks or directories while moving to a new version (like Alpha to Beta transitioning) or reconfiguring (like full re-start).

We will do all our work in the EdgeFS example manifests directory.

  1. cd $YOUR_ROOK_REPO/cluster/examples/kubernetes/edgefs/

Example of migration from existing v1beta1 EdgeFS cluster to stable v1 EdgeFS cluster: For already existing EdgeFS manifests we should replace v1beta1 version to v1. We can use a few simple sed commands to do this for all manifests at once.

  1. sed -i.bak -e "s/edgefs.rook.io\/v1beta1/edgefs.rook.io\/v1beta1/g" *.yaml
  2. sed -i -e "s/edgefs.rook.io\/v1beta1/edgefs.rook.io\/v1/g" *.yaml
  3. mkdir -p backups
  4. mv *.bak backups/

Then we should add new v1 version specification to existing EdgeFS’ CRDs.

  1. kubectl apply -f upgrade-from-v1beta1-create.yaml

And replace EdgeFS operator to new stable ‘v1’ version

  1. kubectl -n rook-edgefs-system set image deploy/rook-edgefs-operator rook-edgefs-operator=rook/edgefs:v1.1.0

Then you could update your cluster’s EdgeFS image for latest one as discribed below.

EdgeFS Version Upgrade

Official EddgeFS container images can be found on Docker Hub.

  1. Update the EdgeFS image used for the EdgeFS cluster:
  1. # Parameterize the environment
  2. export ROOK_SYSTEM_NAMESPACE="rook-edgefs-system"
  3. export CLUSTER_NAME="rook-edgefs"

The majority of the upgrade will be handled by the Rook operator. Begin the upgrade by changing the EdgeFS image field in the cluster CRD (spec:edgefsImageName).

  1. NEW_EDGEFS_IMAGE='edgefs/edgefs:latest'
  2. kubectl -n $CLUSTER_NAME patch Cluster $CLUSTER_NAME --type=merge \
  3.   -p "{\"spec\": {\"edgefsImageName\": \"$NEW_EDGEFS_IMAGE\"}}"

or via console editor update the edgefsImageName property:

  1. kubectl edit -n $CLUSTER_NAME Cluster $CLUSTER_NAME

and save results.

  1. Wait for the pod updates to complete:

As with upgrading Rook, you must now wait for the upgrade to complete. Determining when the EdgeFS version has fully updated is rather simple.

  1. kubectl -n $CLUSTER_NAME describe pods | grep "Image:" | sort | uniq
  2. # This cluster is not yet finished:
  3. #      Image:         edgefs/edgefs:1.2.31
  4. #      Image:         edgefs/edgefs:1.2.50
  5. #      Image:         edgefs/edgefs-restapi:1.2.31
  6. #      Image:         edgefs/edgefs-ui:1.2.31
  7. # This cluster is also finished(all versions are the same):
  8. #      Image:         edgefs/edgefs:1.2.50
  9. #      Image:         edgefs/edgefs-restapi:1.2.50
  10. #      Image:         edgefs/edgefs-ui:1.2.50
  1. Verify the updated EdgeFS cluster:

Access to EdgeFS mgr pod and check EdgeFS system status

  1. kubectl exec -it -n $CLUSTER_NAME rook-edgefs-mgr-xxxx-xxx -- toolbox
  2. efscli system status -v 1

EdgeFS Nodes Update

Nodes can be added and removed over time by updating the Cluster CRD, for example with kubectl edit Cluster -n rook-edgefs. This will bring up your default text editor and allow you to add and remove storage nodes from the cluster. This feature is only available when useAllNodes has been set to false and resurrect mode is not used.

Add node Example

  1. Edit Cluster CRD kubectl edit Cluster -n rook-edgefs
  2. Add new node section with desired configuration in storage section of Cluster CRD:

Currently we adding new node node3072ub16 with two drives sdb and sdc on it.

  1.     - config: null
  2.       devices:
  3.       - FullPath: ""
  4.         config: null
  5.         name: sdb
  6.       - FullPath: ""
  7.         config: null
  8.         name: sdc
  9.       name: node3072ub16
  10.       resources: {}
  • Save CRD and operator will update all target nodes and related pods of the EdgeFS cluster.
  • Login to EdgeFS mgr toolbox and adjust FlexHash table to a new configuration using efscli system fhtable command.

EdgeFS Operator Upgrade

In terms of to bring new Rook Operator features on a running system, it needs to be upgraded first. And once upgraded, you will need to create new CustomResourceDefinition(s).

  1. Edit deployment with kubectl edit deploy rook-edgefs-operator -n rook-edgefs-system and set to a new version
  2. Apply new CRDs or changes. Look at examples in cluster/examples/kubernetes/edgefs/operator.yaml.
  3. Monitor operator logs with kubectl logs rook-edgefs-operator-ID -n rook-edgefs-system