Persistent Storage Using OpenStack Manila

Overview

Persistent volume (PV) provisioning using OpenStack Manila is a Technology Preview feature only.

OKD is capable of provisioning PVs using the OpenStack Manila shared file system service.

It is assumed the OpenStack Manila service has been correctly set up and is accessible from the OKD cluster. Only the NFS share types can be provisioned.

Familiarity with PVs, persistent volume claims (PVCs), dynamic provisioning, and RBAC authorization is recommended.

Installation and Setup

The feature is provided by an external provisioner. You must install and configure it in the OKD cluster.

Starting the External Provisioner

The external provisioner service is distributed as a container image and can be run in the OKD cluster as usual.

To allow the containers managing the API objects, configure the required role-based access control (RBAC) rules as a cluster administrator:

  1. Create a ServiceAccount:

    1. apiVersion: v1
    2. kind: ServiceAccount
    3. metadata:
    4.   name: manila-provisioner-runner

  2. Create a ClusterRole:

    1. kind: ClusterRole
    2. apiVersion: rbac.authorization.k8s.io/v1
    3. metadata:
    4.   name: manila-provisioner-role
    5. rules:
    6.   - apiGroups: [""]
    7.     resources: ["persistentvolumes"]
    8.     verbs: ["get", "list", "watch", "create", "delete"]
    9.   - apiGroups: [""]
    10.     resources: ["persistentvolumeclaims"]
    11.     verbs: ["get", "list", "watch", "update"]
    12.   - apiGroups: ["storage.k8s.io"]
    13.     resources: ["storageclasses"]
    14.     verbs: ["get", "list", "watch"]
    15.   - apiGroups: [""]
    16.     resources: ["events"]
    17.     verbs: ["list", "watch", "create", "update", "patch"]

  3. Bind the rules via ClusterRoleBinding:

    1. apiVersion: rbac.authorization.k8s.io/v1
    2. kind: ClusterRoleBinding
    3. metadata:
    4.   name: manila-provisioner
    5. roleRef:
    6.   apiGroup: rbac.authorization.k8s.io
    7.   kind: ClusterRole
    8.   name: manila-provisioner-role
    9. subjects:
    10. - kind: ServiceAccount
    11.   name: manila-provisioner-runner
    12.   namespace: default

  4. Create a new StorageClass:

    1. apiVersion: storage.k8s.io/v1
    2. kind: StorageClass
    3. metadata:
    4.   name: "manila-share"
    5. provisioner: "externalstorage.k8s.io/manila"
    6. parameters:
    7.   type: "default" (1)
    8.   zones: "nova" (2)
    1The Manila share type the provisioner will create for the volume.
    2Set of Manila availability zones that the volume might be created in.

Configure the provisioner to connect, authenticate, and authorize to the Manila servic using environment variables. Select the appropriate combination of environment variables for your installation from the following list:

  1. OS_USERNAME
  2. OS_PASSWORD
  3. OS_AUTH_URL
  4. OS_DOMAIN_NAME
  5. OS_TENANT_NAME
  1. OS_USERID
  2. OS_PASSWORD
  3. OS_AUTH_URL
  4. OS_TENANT_ID
  1. OS_USERNAME
  2. OS_PASSWORD
  3. OS_AUTH_URL
  4. OS_DOMAIN_ID
  5. OS_TENANT_NAME
  1. OS_USERNAME
  2. OS_PASSWORD
  3. OS_AUTH_URL
  4. OS_DOMAIN_ID
  5. OS_TENANT_ID

To pass the variables to the provisioner, use a Secret. The following example shows a Secret configured for the first variables combination

  1. apiVersion: v1
  2. kind: Secret
  3. metadata:
  4.   name: manila-provisioner-env
  5. type: Opaque
  6. data:
  7.   os_username: <base64 encoded Manila username>
  8.   os_password: <base64 encoded password>
  9.   os_auth_url: <base64 encoded OpenStack Keystone URL>
  10.   os_domain_name: <base64 encoded Manila service Domain>
  11.   os_tenant_name: <base64 encoded Manila service Tenant/Project name>

Newer OpenStack versions use “project” instead of “tenant.” However, the environment variables used by the provisioner must use TENANT in their names.

The last step is to start the provisioner itself, for example, using a deployment:

  1. kind: Deployment
  2. apiVersion: extensions/v1beta1
  3. metadata:
  4.   name: manila-provisioner
  5. spec:
  6.   replicas: 1
  7.   strategy:
  8.     type: Recreate
  9.   template:
  10.     metadata:
  11.       labels:
  12.         app: manila-provisioner
  13.     spec:
  14.       serviceAccountName: manila-provisioner-runner
  15.       containers:
  16.         - image: "registry.redhat.io/openshift3/manila-provisioner:latest"
  17.           imagePullPolicy: "IfNotPresent"
  18.           name: manila-provisioner
  19.           env:
  20.             - name: "OS_USERNAME"
  21.               valueFrom:
  22.                 secretKeyRef:
  23.                   name: manila-provisioner-env
  24.                   key: os_username
  25.             - name: "OS_PASSWORD"
  26.               valueFrom:
  27.                 secretKeyRef:
  28.                   name: manila-provisioner-env
  29.                   key: os_password
  30.             - name: "OS_AUTH_URL"
  31.               valueFrom:
  32.                 secretKeyRef:
  33.                   name: manila-provisioner-env
  34.                   key: os_auth_url
  35.             - name: "OS_DOMAIN_NAME"
  36.               valueFrom:
  37.                 secretKeyRef:
  38.                   name: manila-provisioner-env
  39.                   key: os_domain_name
  40.             - name: "OS_TENANT_NAME"
  41.               valueFrom:
  42.                 secretKeyRef:
  43.                   name: manila-provisioner-env
  44.                   key: os_tenant_name

Usage

After the provisioner is running, you can provision PVs using a PVC and the corresponding StorageClass:

  1. kind: PersistentVolumeClaim
  2. apiVersion: v1
  3. metadata:
  4.   name: manila-nfs-pvc
  5. spec:
  6.   accessModes:
  7.     - ReadWriteOnce
  8.   resources:
  9.     requests:
  10.       storage: 2G
  11.   storageClassName: manila-share

The PersistentVolumeClaim is then bound to a PersistentVolume backed by the newly provisioned Manila share. When the PersistentVolumeClaim and subsequently the PersistentVolume are deleted, the provisioner deletes and unexports the Manila share.