Persistent storage using FlexVolume

OKD supports FlexVolume, an out-of-tree plug-in that uses an executable model to interface with drivers.

To use storage from a back-end that does not have a built-in plug-in, you can extend OKD through FlexVolume drivers and provide persistent storage to applications.

Pods interact with FlexVolume drivers through the flexvolume in-tree plugin.

Additional resources

• Expanding persistent volumes

About FlexVolume drivers

A FlexVolume driver is an executable file that resides in a well-defined directory on all nodes in the cluster. OKD calls the FlexVolume driver whenever it needs to mount or unmount a volume represented by a PersistentVolume object with flexVolume as the source.

FlexVolume driver example

The first command-line argument of the FlexVolume driver is always an operation name. Other parameters are specific to each operation. Most of the operations take a JavaScript Object Notation (JSON) string as a parameter. This parameter is a complete JSON string, and not the name of a file with the JSON data.

The FlexVolume driver contains:

• All flexVolume.options.

• Some options from flexVolume prefixed by kubernetes.io/, such as fsType and readwrite.

• The content of the referenced secret, if specified, prefixed by kubernetes.io/secret/.

FlexVolume driver JSON input example

{
    "fooServer": "192.168.0.1:1234", (1)
        "fooVolumeName": "bar",
    "kubernetes.io/fsType": "ext4", (2)
    "kubernetes.io/readwrite": "ro", (3)
    "kubernetes.io/secret/<key name>": "<key value>", (4)
    "kubernetes.io/secret/<another key name>": "<another key value>",
}

OKD expects JSON data on standard output of the driver. When not specified, the output describes the result of the operation.

FlexVolume driver default output example

{
    "status": "<Success/Failure/Not supported>",
    "message": "<Reason for success/failure>"
}

Exit code of the driver should be 0 for success and 1 for error.

Operations should be idempotent, which means that the mounting of an already mounted volume should result in a successful operation.

Installing FlexVolume drivers

FlexVolume drivers that are used to extend OKD are executed only on the node. To implement FlexVolumes, a list of operations to call and the installation path are all that is required.

Prerequisites

• FlexVolume drivers must implement these operations:

  init

  Initializes the driver. It is called during initialization of all nodes.

  • Arguments: none

  • Executed on: node

  • Expected output: default JSON

  mount

  Mounts a volume to directory. This can include anything that is necessary to mount the volume, including finding the device and then mounting the device.

  • Arguments: <mount-dir> <json>

  • Executed on: node

  • Expected output: default JSON

  unmount

  Unmounts a volume from a directory. This can include anything that is necessary to clean up the volume after unmounting.

  • Arguments: <mount-dir>

  • Executed on: node

  • Expected output: default JSON

  mountdevice

  Mounts a volume’s device to a directory where individual pods can then bind mount.

This call-out does not pass “secrets” specified in the FlexVolume spec. If your driver requires secrets, do not implement this call-out.

• Arguments: <mount-dir> <json>

• Executed on: node

• Expected output: default JSON

  unmountdevice

  Unmounts a volume’s device from a directory.

• Arguments: <mount-dir>

• Executed on: node

• Expected output: default JSON

  • All other operations should return JSON with {"status": "Not supported"} and exit code 1.

Procedure

To install the FlexVolume driver:

1. Ensure that the executable file exists on all nodes in the cluster.

2. Place the executable file at the volume plug-in path: /etc/kubernetes/kubelet-plugins/volume/exec/<vendor>~<driver>/<driver>.

For example, to install the FlexVolume driver for the storage foo, place the executable file at: /etc/kubernetes/kubelet-plugins/volume/exec/openshift.com~foo/foo.

Consuming storage using FlexVolume drivers

Each PersistentVolume object in OKD represents one storage asset in the storage back-end, such as a volume.

Procedure

• Use the PersistentVolume object to reference the installed storage.

Persistent volume object definition using FlexVolume drivers example

apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv0001 (1)
spec:
  capacity:
    storage: 1Gi (2)
  accessModes:
    - ReadWriteOnce
  flexVolume:
    driver: openshift.com/foo (3)
    fsType: "ext4" (4)
    secretRef: foo-secret (5)
    readOnly: true (6)
    options: (7)
      fooServer: 192.168.0.1:1234
      fooVolumeName: bar

“fsType”:”<FS type>”,
“readwrite”:”<rw>”,
“secret/key1”:”<secret1>”
…
“secret/keyN”:”<secretN>”