Persistent storage using iSCSI

You can provision your OKD cluster with persistent storage using iSCSI. Some familiarity with Kubernetes and iSCSI is assumed.

The Kubernetes persistent volume framework allows administrators to provision a cluster with persistent storage and gives users a way to request those resources without having any knowledge of the underlying infrastructure.

High-availability of storage in the infrastructure is left to the underlying storage provider.

When you use iSCSI on Amazon Web Services, you must update the default security policy to include TCP traffic between nodes on the iSCSI ports. By default, they are ports 860 and 3260.

OpenShift assumes that all nodes in the cluster have already configured iSCSI initator, i.e. have installed iscsi-initiator-utils package and configured their initiator name in /etc/iscsi/initiatorname.iscsi. See Storage Administration Guide linked above.

Provisioning

Verify that the storage exists in the underlying infrastructure before mounting it as a volume in OKD. All that is required for the iSCSI is the iSCSI target portal, a valid iSCSI Qualified Name (IQN), a valid LUN number, the filesystem type, and the PersistentVolume API.

PersistentVolume object definition

  1. apiVersion: v1
  2. kind: PersistentVolume
  3. metadata:
  4.   name: iscsi-pv
  5. spec:
  6.   capacity:
  7.     storage: 1Gi
  8.   accessModes:
  9.     - ReadWriteOnce
  10.   iscsi:
  11.      targetPortal: 10.16.154.81:3260
  12.      iqn: iqn.2014-12.example.server:storage.target00
  13.      lun: 0
  14.      fsType: 'ext4'

Enforcing disk quotas

Use LUN partitions to enforce disk quotas and size constraints. Each LUN is one persistent volume. Kubernetes enforces unique names for persistent volumes.

Enforcing quotas in this way allows the end user to request persistent storage by a specific amount (e.g, 10Gi) and be matched with a corresponding volume of equal or greater capacity.

iSCSI volume security

Users request storage with a PersistentVolumeClaim object. This claim only lives in the user’s namespace and can only be referenced by a pod within that same namespace. Any attempt to access a persistent volume claim across a namespace causes the pod to fail.

Each iSCSI LUN must be accessible by all nodes in the cluster.

Challenge Handshake Authentication Protocol (CHAP) configuration

Optionally, OpenShift can use CHAP to authenticate itself to iSCSI targets:

  1. apiVersion: v1
  2. kind: PersistentVolume
  3. metadata:
  4.   name: iscsi-pv
  5. spec:
  6.   capacity:
  7.     storage: 1Gi
  8.   accessModes:
  9.     - ReadWriteOnce
  10.   iscsi:
  11.     targetPortal: 10.0.0.1:3260
  12.     iqn: iqn.2016-04.test.com:storage.target00
  13.     lun: 0
  14.     fsType: ext4
  15.     chapAuthDiscovery: true (1)
  16.     chapAuthSession: true (2)
  17.     secretRef:
  18.       name: chap-secret (3)
1Enable CHAP authentication of iSCSI discovery.
2Enable CHAP authentication of iSCSI session.
3Specify name of Secrets object with user name + password. This Secret object must be available in all namespaces that can use the referenced volume.

iSCSI multipathing

For iSCSI-based storage, you can configure multiple paths by using the same IQN for more than one target portal IP address. Multipathing ensures access to the persistent volume when one or more of the components in a path fail.

To specify multi-paths in the pod specification use the portals field. For example:

  1. apiVersion: v1
  2. kind: PersistentVolume
  3. metadata:
  4.   name: iscsi-pv
  5. spec:
  6.   capacity:
  7.     storage: 1Gi
  8.   accessModes:
  9.     - ReadWriteOnce
  10.   iscsi:
  11.     targetPortal: 10.0.0.1:3260
  12.     portals: ['10.0.2.16:3260', '10.0.2.17:3260', '10.0.2.18:3260'] (1)
  13.     iqn: iqn.2016-04.test.com:storage.target00
  14.     lun: 0
  15.     fsType: ext4
  16.     readOnly: false
1Add additional target portals using the portals field.

iSCSI custom initiator IQN

Configure the custom initiator iSCSI Qualified Name (IQN) if the iSCSI targets are restricted to certain IQNs, but the nodes that the iSCSI PVs are attached to are not guaranteed to have these IQNs.

To specify a custom initiator IQN, use initiatorName field.

  1. apiVersion: v1
  2. kind: PersistentVolume
  3. metadata:
  4.   name: iscsi-pv
  5. spec:
  6.   capacity:
  7.     storage: 1Gi
  8.   accessModes:
  9.     - ReadWriteOnce
  10.   iscsi:
  11.     targetPortal: 10.0.0.1:3260
  12.     portals: ['10.0.2.16:3260', '10.0.2.17:3260', '10.0.2.18:3260']
  13.     iqn: iqn.2016-04.test.com:storage.target00
  14.     lun: 0
  15.     initiatorName: iqn.2016-04.test.com:custom.iqn (1)
  16.     fsType: ext4
  17.     readOnly: false
1Specify the name of the initiator.