Persistent storage using AWS Elastic Block Store

OKD supports AWS Elastic Block Store volumes (EBS). You can provision your OKD cluster with persistent storage by using Amazon EC2. Some familiarity with Kubernetes and AWS is assumed.

The Kubernetes persistent volume framework allows administrators to provision a cluster with persistent storage and gives users a way to request those resources without having any knowledge of the underlying infrastructure. AWS Elastic Block Store volumes can be provisioned dynamically. Persistent volumes are not bound to a single project or namespace; they can be shared across the OKD cluster. Persistent volume claims are specific to a project or namespace and can be requested by users. You can define a KMS key to encrypt container-persistent volumes on AWS.

For OKD, automatic migration from AWS EBS in-tree to the Container Storage Interface (CSI) driver is available as a Technology Preview (TP) feature. With migration enabled, volumes provisioned using the existing in-tree driver are automatically migrated to use the AWS EBS CSI driver. For more information, see CSI automatic migration feature.

Creating the EBS storage class

Storage classes are used to differentiate and delineate storage levels and usages. By defining a storage class, users can obtain dynamically provisioned persistent volumes.

Procedure

1. In the OKD console, click Storage → Storage Classes.

2. On the StorageClasses overview page, click Create Storage Class.

3. On the StorageClasses create page, enter values as desired:

   1. Enter a name to reference the storage class.

   2. Enter an optional description.

   3. Select the reclaim policy.

   4. Select kubernetes.io/aws-ebs from the Provisioner drop-down list.

      To create the storage class with the equivalent CSI driver, select ebs.csi.aws.com from the drop-down list. For more details, see AWS Elastic Block Store CSI Driver Operator.

   5. Enter additional parameters for the storage class as desired.

4. Click Create.

Creating the persistent volume claim

Prerequisites

Storage must exist in the underlying infrastructure before it can be mounted as a volume in OKD.

Procedure

1. In the OKD console, click Storage → Persistent Volume Claims.

2. In the persistent volume claims overview, click Create Persistent Volume Claim.

3. Define the desired options on the page that appears.

   1. Select the storage class created previously from the drop-down menu.

   2. Enter a unique name for the storage claim.

   3. Select the access mode. This determines the read and write access for the created storage claim.

   4. Define the size of the storage claim.

4. Click Create to create the persistent volume claim and generate a persistent volume.

Volume format

Before OKD mounts the volume and passes it to a container, it checks that it contains a file system as specified by the fsType parameter in the persistent volume definition. If the device is not formatted with the file system, all data from the device is erased and the device is automatically formatted with the given file system.

This allows using unformatted AWS volumes as persistent volumes, because OKD formats them before the first use.

Maximum number of EBS volumes on a node

By default, OKD supports a maximum of 39 EBS volumes attached to one node. This limit is consistent with the AWS volume limits. The volume limit depends on the instance type.

Encrypting container persistent volumes on AWS with a KMS key

Defining a KMS key to encrypt container-persistent volumes on AWS is useful when you have explicit compliance and security guidelines when deploying to AWS.

Prerequisites

• Underlying infrastructure must contain storage.

• You must create a customer KMS key on AWS.

Procedure

1. Create a storage class:

   $ cat << EOF | oc create -f -
   apiVersion: storage.k8s.io/v1
   kind: StorageClass
   metadata:
     name: <storage-class-name> (1)
   parameters:
     fsType: ext4 (2)
     encrypted: "true"
     kmsKeyId: keyvalue (3)
   provisioner: ebs.csi.aws.com
   reclaimPolicy: Delete
   volumeBindingMode: WaitForFirstConsumer
   EOF

   1	Specifies the name of the storage class.
   2	File system that is created on provisioned volumes.
   3	Specifies the full Amazon Resource Name (ARN) of the key to use when encrypting the container-persistent volume. If you do not provide any key, but the encrypted field is set to true, then the default KMS key is used. See Finding the key ID and key ARN on AWS in the AWS documentation.

2. Create a persistent volume claim (PVC) with the storage class specifying the KMS key:

   $ cat << EOF | oc create -f -
   apiVersion: v1
   kind: PersistentVolumeClaim
   metadata:
     name: mypvc
   spec:
     accessModes:
       - ReadWriteOnce
     volumeMode: Filesystem
     storageClassName: <storage-class-name>
     resources:
       requests:
         storage: 1Gi
   EOF

3. Create workload containers to consume the PVC:

   $ cat << EOF | oc create -f -
   kind: Pod
   metadata:
     name: mypod
   spec:
     containers:
       - name: httpd
         image: quay.io/centos7/httpd-24-centos7
         ports:
           - containerPort: 80
         volumeMounts:
           - mountPath: /mnt/storage
             name: data
     volumes:
       - name: data
         persistentVolumeClaim:
           claimName: mypvc
   EOF

Additional resources

• See AWS Elastic Block Store CSI Driver Operator for information about accessing additional storage options, such as volume snapshots, that are not possible with in-tree volume plug-ins.