Using VMware vSphere volumes for persistent storage

Overview

OKD supports VMware vSphere’s Virtual Machine Disk (VMDK) volumes. You can provision your OKD cluster with persistent storage using VMware vSphere. Some familiarity with Kubernetes and VMware vSphere is assumed.

OKD creates the disk in vSphere and attaches the disk to the correct instance.

The OKD persistent volume (PV) framework allows administrators to provision a cluster with persistent storage and gives users a way to request those resources without having any knowledge of the underlying infrastructure. vSphere VMDK volumes can be provisioned dynamically.

PVs are not bound to a single project or namespace; they can be shared across the OKD cluster. PV claims, however, are specific to a project or namespace and can be requested by users.

Prerequisites

Before creating PVs using vSphere, ensure your OKD cluster meets the following requirements:

• OKD must first be configured for vSphere.

• Each node host in the infrastructure must match the vSphere VM name.

• Each node host must be in the same resource group.

Dynamically Provisioning VMware vSphere volumes

Dynamically provisioning VMware vSphere volumes is the preferred provisioning method.

1. If you did not specify the openshift_cloudprovider_kind=vsphere and openshift_vsphere_* variables in the Ansible inventory file when you provisioned the cluster, you must manually create the following StorageClass to use the vsphere-volume provisioner:

   $ oc get --export storageclass vsphere-standard -o yaml
   kind: StorageClass
   apiVersion: storage.k8s.io/v1
   metadata:
     name: "vsphere-standard" (1)
   provisioner: kubernetes.io/vsphere-volume (2)
   parameters:
       diskformat: thin (3)
       datastore: "YourvSphereDatastoreName" (4)
   reclaimPolicy: Delete

   1	The name of the StorageClass.
   2	The type of storage provisioner. Specify vsphere-volume.
   3	The type of disk. Specify either zeroedthick or thin.
   4	The source datastore where the disks will be created.

2. After you request a PV, using the StorageClass shown in the previous step, OKD automatically creates VMDK disks in the vSphere infrastructure. To verify that the disks were created, use the Datastore browser in vSphere.

   vSphere-volume disks are ReadWriteOnce access mode, which means the volume can be mounted as read-write by a single node. See the Access modes section of the Architecture guide for more information.

Statically Provisioning VMware vSphere volumes

Storage must exist in the underlying infrastructure before it can be mounted as a volume in OKD. After ensuring OKD is configured for vSphere, all that is required for OKD and vSphere is a VM folder path, file system type, and the PersistentVolume API.

Create the VMDKs

• Create using vmkfstools:

  Access ESX through Secure Shell (SSH) and then use following command to create a VMDK volume:

  vmkfstools -c 40G /vmfs/volumes/DatastoreName/volumes/myDisk.vmdk

• Create using vmware-vdiskmanager:

  shell vmware-vdiskmanager -c -t 0 -s 40GB -a lsilogic myDisk.vmdk

Creating PersistentVolumes

1. Define a PV object definition, for example vsphere-pv.yaml:

   apiVersion: v1
   kind: PersistentVolume
   metadata:
     name: pv0001 (1)
   spec:
     capacity:
       storage: 2Gi (2)
     accessModes:
       - ReadWriteOnce
     persistentVolumeReclaimPolicy: Retain
     vsphereVolume: (3)
       volumePath: "[datastore1] volumes/myDisk" (4)
       fsType: ext4 (5)

   1	The name of the volume. This must be how it is identified by PV claims or from pods.
   2	The amount of storage allocated to this volume.
   3	The volume type being used. This example uses vsphereVolume. The label is used to mount a vSphere VMDK volume into pods. The contents of a volume are preserved when it is unmounted. The volume type supports VMFS and VSAN datastore.
   4	The existing VMDK volume to use. You must enclose the datastore name in square brackets ([]) in the volume definition, as shown.
   5	The file system type to mount. For example, ext4, xfs, or other file-systems.

   Changing the value of the fsType parameter after the volume is formatted and provisioned can result in data loss and pod failure.

2. Create the PV:

   $ oc create -f vsphere-pv.yaml
     persistentvolume "pv0001" created

3. Verify that the PV was created:

   $ oc get pv
   NAME    LABELS  CAPACITY  ACCESSMODES   STATUS    CLAIM   REASON  AGE
   pv0001  <none>  2Gi       RWO           Available                 2s

Now you can request storage using PV claims, which can now use your PV.

Formatting VMware vSphere volumes

Before OKD mounts the volume and passes it to a container, it checks that the volume contains a file system as specified by the fsType parameter in the PV definition. If the device is not formatted with the file system, all data from the device is erased, and the device is automatically formatted with the given file system.

Because OKD formats them before the first use, you can use unformatted vSphere volumes as PVs.