Dynamic provisioning and creating storage classes

Overview

The StorageClass resource object describes and classifies storage that can be requested, as well as provides a means for passing parameters for dynamically provisioned storage on demand. StorageClass objects can also serve as a management mechanism for controlling different levels of storage and access to the storage. Cluster Administrators (cluster-admin) or Storage Administrators (storage-admin) define and create the StorageClass objects that users can request without needing any intimate knowledge about the underlying storage volume sources.

The OKD persistent volume framework enables this functionality and allows administrators to provision a cluster with persistent storage. The framework also gives users a way to request those resources without having any knowledge of the underlying infrastructure.

Many storage types are available for use as persistent volumes in OKD. While all of them can be statically provisioned by an administrator, some types of storage are created dynamically using the built-in provider and plug-in APIs.

[OSEv3:vars]
openshift_master_dynamic_provisioning_enabled=True

Available dynamically provisioned plug-ins

OKD provides the following provisioner plug-ins, which have generic implementations for dynamic provisioning that use the cluster’s configured provider’s API to create new storage resources:

Defining a StorageClass

StorageClass objects are currently a globally scoped object and need to be created by cluster-admin or storage-admin users.

There are currently six plug-ins that are supported. The following sections describe the basic object definition for a StorageClass and specific examples for each of the supported plug-in types.

Basic StorageClass object definition

StorageClass Basic object definition

kind: StorageClass (1)
apiVersion: storage.k8s.io/v1 (2)
metadata:
  name: foo (3)
  annotations: (4)
     ...
provisioner: kubernetes.io/plug-in-type (5)
parameters: (6)
  param1: value
  ...
  paramN: value

StorageClass annotations

To set a StorageClass as the cluster-wide default:

   storageclass.kubernetes.io/is-default-class: "true"

This enables any Persistent Volume Claim (PVC) that does not specify a specific volume to automatically be provisioned through the default StorageClass

To set a StorageClass description:

   kubernetes.io/description: My StorageClass Description

OpenStack Cinder object definition

cinder-storageclass.yaml

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: gold
provisioner: kubernetes.io/cinder
parameters:
  type: fast  (1)
  availability: nova (2)
  fsType: ext4 (3)

AWS ElasticBlockStore (EBS) object definition

aws-ebs-storageclass.yaml

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: slow
provisioner: kubernetes.io/aws-ebs
parameters:
  type: io1 (1)
  zone: us-east-1d (2)
  iopsPerGB: "10" (3)
  encrypted: "true" (4)
  kmsKeyId: keyvalue (5)
  fsType: ext4 (6)

GCE PersistentDisk (gcePD) object definition

gce-pd-storageclass.yaml

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: slow
provisioner: kubernetes.io/gce-pd
parameters:
  type: pd-standard  (1)
  zone: us-central1-a  (2)
  zones: us-central1-a, us-central1-b, us-east1-b  (3)
  fsType: ext4 (4)

GlusterFS object definition

glusterfs-storageclass.yaml

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: slow
provisioner: kubernetes.io/glusterfs
parameters: (1)
  resturl: http://127.0.0.1:8081 (2)
  restuser: admin (3)
  secretName: heketi-secret (4)
  secretNamespace: default (5)
  gidMin: "40000" (6)
  gidMax: "50000" (7)
  volumeoptions: group metadata-cache, nl-cache on (8)
  volumetype: replicate:3 (9)
  volumenameprefix: custom (10)

When heketi authentication is used, a Secret containing the admin key should also exist:

heketi-secret.yaml

apiVersion: v1
kind: Secret
metadata:
  name: heketi-secret
  namespace: default
data:
  key: bXlwYXNzd29yZA== (1)
type: kubernetes.io/glusterfs

Ceph RBD object definition

ceph-storageclass.yaml

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: fast
provisioner: kubernetes.io/rbd
parameters:
  monitors: 10.16.153.105:6789  (1)
  adminId: admin  (2)
  adminSecretName: ceph-secret  (3)
  adminSecretNamespace: kube-system  (4)
  pool: kube  (5)
  userId: kube  (6)
  userSecretName: ceph-secret-user  (7)
  fsType: ext4 (8)

Trident object definition

trident.yaml

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: gold
provisioner: netapp.io/trident (1)
parameters: (2)
  media: "ssd"
  provisioningType: "thin"
  snapshots: "true"

Trident uses the parameters as selection criteria for the different pools of storage that are registered with it. Trident itself is configured separately.

VMware vSphere object definition

vsphere-storageclass.yaml

kind: StorageClass
apiVersion: storage.k8s.io/v1beta1
metadata:
  name: slow
provisioner: kubernetes.io/vsphere-volume (1)
parameters:
  diskformat: thin (2)

Azure File object definition

To configure Azure file dynamic provisioning:

1. Create the role in the user’s project:

   $ cat azf-role.yaml
   apiVersion: rbac.authorization.k8s.io/v1
   kind: Role
   metadata:
     name: system:controller:persistent-volume-binder
     namespace: <user's project name>
   rules:
     - apiGroups: [""]
       resources: ["secrets"]
       verbs: ["create", "get", "delete"]

2. Create the role binding to the persistent-volume-binder service account in the kube-system project:

   $ cat azf-rolebind.yaml
   apiVersion: rbac.authorization.k8s.io/v1
   kind: RoleBinding
   metadata:
     name: system:controller:persistent-volume-binder
     namespace: <user's project>
   roleRef:
     apiGroup: rbac.authorization.k8s.io
     kind: Role
     name: system:controller:persistent-volume-binder
   subjects:
   - kind: ServiceAccount
     name: persistent-volume-binder
   namespace: kube-system

3. Add the service account as admin to the user’s project:

   $ oc policy add-role-to-user admin system:serviceaccount:kube-system:persistent-volume-binder -n <user's project>

4. Create a storage class for the Azure file:

   $ cat azfsc.yaml | oc create -f -
   kind: StorageClass
   apiVersion: storage.k8s.io/v1
   metadata:
     name: azfsc
   provisioner: kubernetes.io/azure-file
   mountOptions:
     - dir_mode=0777
     - file_mode=0777

The user can now create a PVC that uses this storage class.

Azure Disk object definition

azure-advanced-disk-storageclass.yaml

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: slow
provisioner: kubernetes.io/azure-disk
parameters:
  storageAccount: azure_storage_account_name  (1)
  storageaccounttype: Standard_LRS  (2)
  kind: Dedicated  (3)

Changing the default StorageClass

If you are using GCE and AWS, use the following process to change the default StorageClass:

1. List the StorageClass:

   $ oc get storageclass
   NAME                 TYPE
   gp2 (default)        kubernetes.io/aws-ebs (1)
   standard             kubernetes.io/gce-pd

   1	(default) denotes the default StorageClass.

2. Change the value of the annotation storageclass.kubernetes.io/is-default-class to false for the default StorageClass:

   $ oc patch storageclass gp2 -p '{"metadata": {"annotations": \
       {"storageclass.kubernetes.io/is-default-class": "false"}}}'

3. Make another StorageClass the default by adding or modifying the annotation as storageclass.kubernetes.io/is-default-class=true.

   $ oc patch storageclass standard -p '{"metadata": {"annotations": \
       {"storageclass.kubernetes.io/is-default-class": "true"}}}'

1. Verify the changes:

   $ oc get storageclass
   NAME                 TYPE
   gp2                  kubernetes.io/aws-ebs
   standard (default)   kubernetes.io/gce-pd

Additional information and examples

• Examples and uses of StorageClasses for Dynamic Provisioning

• Examples and uses of StorageClasses without Dynamic Provisioning