3 - vSphere Storage

To provide stateful workloads with vSphere storage, we recommend creating a vSphereVolume storage class. This practice dynamically provisions vSphere storage when workloads request volumes through a persistent volume claim.

Prerequisites

In order to provision vSphere volumes in a cluster created with the Rancher Kubernetes Engine (RKE), the vSphere cloud provider must be explicitly enabled in the cluster options.

Creating A Storage Class

Note:

The following steps can also be performed using the kubectl command line tool. See Kubernetes documentation on persistent volumes for details.

  • From the Global view, open the cluster where you want to provide vSphere storage.
  • From the main menu, select Storage > Storage Classes. Then click Add Class.
  • Enter a Name for the class.
  • Under Provisioner, select VMWare vSphere Volume.

vsphere-storage-class

  • Optionally, specify additional properties for this storage class under Parameters. Refer to the vSphere storage documentation for details.

  • Click Save.

Creating a Workload with a vSphere Volume

  • From the cluster where you configured vSphere storage, begin creating a workload as you would in Deploying Workloads.
  • For Workload Type, select Stateful set of 1 pod.
  • Expand the Volumes section and click Add Volume.
  • Choose Add a new persistent volume (claim). This option will implicitly create the claim once you deploy the workload.
  • Assign a Name for the claim, ie. test-volume and select the vSphere storage class created in the previous step.
  • Enter the required Capacity for the volume. Then click Define.

workload-add-volume

  • Assign a path in the Mount Point field. This is the full path where the volume will be mounted in the container file system, e.g. /persistent.

  • Click Launch to create the workload.

Verifing Persistence of the Volume

  • From the context menu of the workload you just created, click Execute Shell.
  • Note the directory at root where the volume has been mounted to (in this case /persistent).
  • Create a file in the volume by executing the command touch /<volumeMountPoint>/data.txt.
  • Close the shell window.
  • Click on the name of the workload to reveal detail information.
  • Open the context menu next to the Pod in the Running state.
  • Delete the Pod by selecting Delete.
  • Observe that the pod is deleted. Then a new pod is scheduled to replace it so that the workload maintains its configured scale of a single stateful pod.
  • Once the replacement pod is running, click Execute Shell.
  • Inspect the contents of the directory where the volume is mounted by entering ls -l /<volumeMountPoint>. Note that the file you created earlier is still present.

workload-persistent-data

Why to Use StatefulSets Instead of Deployments

You should always use StatefulSets for workloads consuming vSphere storage, as this resource type is designed to address a VMDK block storage caveat.

Since vSphere volumes are backed by VMDK block storage, they only support an access mode of ReadWriteOnce. This setting restricts the volume so that it can only be mounted to a single pod at a time, unless all pods consuming that volume are co-located on the same node. This behavior makes a deployment resource unusable for scaling beyond a single replica if it consumes vSphere volumes.

Even using a deployment resource with just a single replica may result in a deadlock situation while updating the deployment. If the updated pod is scheduled to a node different from where the existing pod lives, it will fail to start because the VMDK is still attached to the other node.