Shared Resource CSI Driver Operator

As a cluster administrator, you can use the Shared Resource CSI Driver in OKD to provision inline ephemeral volumes that contain the contents of Secret or ConfigMap objects. This way, pods and other Kubernetes types that expose volume mounts, and OKD Builds can securely use the contents of those objects across potentially any namespace in the cluster. To accomplish this, there are currently two types of shared resources: a SharedSecret custom resource for Secret objects, and a SharedConfigMap custom resource for ConfigMap objects.

About CSI

Storage vendors have traditionally provided storage drivers as part of Kubernetes. With the implementation of the Container Storage Interface (CSI), third-party providers can instead deliver storage plugins using a standard interface without ever having to change the core Kubernetes code.

CSI Operators give OKD users storage options, such as volume snapshots, that are not possible with in-tree volume plugins.

Sharing secrets across namespaces

To share a secret across namespaces in a cluster, you create a SharedSecret custom resource (CR) instance for the Secret object that you want to share.

Prerequisites

You must have permission to perform the following actions:

• Create instances of the sharedsecrets.sharedresource.openshift.io custom resource definition (CRD) at a cluster-scoped level.

• Manage roles and role bindings across the namespaces in the cluster to control which users can get, list, and watch those instances.

• Manage roles and role bindings to control whether the service account specified by a pod can mount a Container Storage Interface (CSI) volume that references the SharedSecret CR instance you want to use.

• Access the namespaces that contain the Secrets you want to share.

Procedure

• Create a SharedSecret CR instance for the Secret object you want to share across namespaces in the cluster:

  $ oc apply -f - <<EOF
  apiVersion: sharedresource.openshift.io/v1alpha1
  kind: SharedSecret
  metadata:
    name: my-share
  spec:
    secretRef:
      name: <name of secret>
      namespace: <namespace of secret>
  EOF

Using a SharedSecret instance in a pod

To access a SharedSecret custom resource (CR) instance from a pod, you grant a given service account RBAC permissions to use that SharedSecret CR instance.

Prerequisites

• You have created a SharedSecret CR instance for the secret you want to share across namespaces in the cluster.

• You must have permission to perform the following actions

  • Discover which SharedSecret CR instances are available by entering the oc get sharedsecrets command and getting a non-empty list back.

  • Determine if the service account your pod specifies is allowed to use the given SharedSecret CR instance. That is, you can run oc adm policy who-can use <identifier of specific SharedSecret> to see if the service account in your namespace is listed.

  • Determine if the service account your pod specifies is allowed to use csi volumes, or if you, as the requesting user who created the pod directly, are allowed to use csi volumes. See “Understanding and managing pod security admission” for details.

Procedure

1. Grant a given service account RBAC permissions to use the SharedSecret CR instance in its pod by using oc apply with YAML content:

   Currently, kubectl and oc have hard-coded special case logic restricting the use verb to roles centered around pod security. Therefore, you cannot use oc create role …​ to create the role needed for consuming SharedSecret CR instances.

   $ oc apply -f - <<EOF
   apiVersion: rbac.authorization.k8s.io/v1
   kind: Role
   metadata:
     name: shared-resource-my-share
     namespace: my-namespace
   rules:
     - apiGroups:
         - sharedresource.openshift.io
       resources:
         - sharedsecrets
       resourceNames:
         - my-share
       verbs:
         - use
   EOF

2. Create the RoleBinding associated with the role by using the oc command:

   $ oc create rolebinding shared-resource-my-share --role=shared-resource-my-share --serviceaccount=my-namespace:builder

3. Access the SharedSecret CR instance from a pod:

   $ oc apply -f - <<EOF
   kind: Pod
   apiVersion: v1
   metadata:
     name: my-app
     namespace: my-namespace
   spec:
     serviceAccountName: default
   # containers omitted …. Follow standard use of ‘volumeMounts’ for referencing your shared resource volume
       volumes:
       - name: my-csi-volume
         csi:
           readOnly: true
           driver: csi.sharedresource.openshift.io
           volumeAttributes:
             sharedSecret: my-share
   EOF

Sharing a config map across namespaces

To share a config map across namespaces in a cluster, you create a SharedConfigMap custom resource (CR) instance for that config map.

Prerequisites

You must have permission to perform the following actions:

• Create instances of the sharedconfigmaps.sharedresource.openshift.io custom resource definition (CRD) at a cluster-scoped level.

• Manage roles and role bindings across the namespaces in the cluster to control which users can get, list, and watch those instances.

• Manage roles and role bindings across the namespaces in the cluster to control which service accounts in pods that mount your Container Storage Interface (CSI) volume can use those instances.

• Access the namespaces that contain the Secrets you want to share.

Procedure

1. Create a SharedConfigMap CR instance for the config map that you want to share across namespaces in the cluster:

   $ oc apply -f - <<EOF
   apiVersion: sharedresource.openshift.io/v1alpha1
   kind: SharedConfigMap
   metadata:
     name: my-share
   spec:
     configMapRef:
       name: <name of configmap>
       namespace: <namespace of configmap>
   EOF

Using a SharedConfigMap instance in a pod

Next steps

To access a SharedConfigMap custom resource (CR) instance from a pod, you grant a given service account RBAC permissions to use that SharedConfigMap CR instance.

Prerequisites

• You have created a SharedConfigMap CR instance for the config map that you want to share across namespaces in the cluster.

• You must have permission to perform the following actions:

  • Discover which SharedConfigMap CR instances are available by entering the oc get sharedconfigmaps command and getting a non-empty list back.

  • Determine if the service account your pod specifies is allowed to use the given SharedSecret CR instance. That is, you can run oc adm policy who-can use <identifier of specific SharedSecret> to see if the service account in your namespace is listed.

  • Determine if the service account your pod specifies is allowed to use csi volumes, or if you, as the requesting user who created the pod directly, are allowed to use csi volumes. See “Understanding and managing pod security admission” for details.

Procedure

1. Grant a given service account RBAC permissions to use the SharedConfigMap CR instance in its pod by using oc apply with YAML content.

   Currently, kubectl and oc have hard-coded special case logic restricting the use verb to roles centered around pod security. Therefore, you cannot use oc create role …​ to create the role needed for consuming a SharedConfigMap CR instance.

   $ oc apply -f - <<EOF
   apiVersion: rbac.authorization.k8s.io/v1
   kind: Role
   metadata:
     name: shared-resource-my-share
     namespace: my-namespace
   rules:
     - apiGroups:
         - sharedresource.openshift.io
       resources:
         - sharedconfigmaps
       resourceNames:
         - my-share
       verbs:
         - use
   EOF

2. Create the RoleBinding associated with the role by using the oc command:

   oc create rolebinding shared-resource-my-share --role=shared-resource-my-share --serviceaccount=my-namespace:builder

3. Access the SharedConfigMap CR instance from a pod:

   $ oc apply -f - <<EOF
   kind: Pod
   apiVersion: v1
   metadata:
     name: my-app
     namespace: my-namespace
   spec:
     serviceAccountName: default
   # containers omitted …. Follow standard use of ‘volumeMounts’ for referencing your shared resource volume
       volumes:
       - name: my-csi-volume
         csi:
           readOnly: true
           driver: csi.sharedresource.openshift.io
           volumeAttributes:
             sharedConfigMap: my-share
   EOF

Additional support limitations for the Shared Resource CSI Driver

The Shared Resource CSI Driver has the following noteworthy limitations:

• The driver is subject to the limitations of Container Storage Interface (CSI) inline ephemeral volumes.

• The value of the readOnly field must be true. On Pod creation, a validating admission webhook rejects the pod creation if readOnly is false. If for some reason the validating admission webhook cannot be contacted, on volume provisioning during pod startup, the driver returns an error to the kubelet. Requiring readOnly is true is in keeping with proposed best practices for the upstream Kubernetes CSI Driver to apply SELinux labels to associated volumes.

• The driver ignores the FSType field because it only supports tmpfs volumes.

• The driver ignores the NodePublishSecretRef field. Instead, it uses SubjectAccessReviews with the use verb to evaluate whether a pod can obtain a volume that contains SharedSecret or SharedConfigMap custom resource (CR) instances.

• You cannot create SharedSecret or SharedConfigMap custom resource (CR) instances whose names start with openshift.

Additional details about VolumeAttributes on shared resource pod volumes

The following attributes affect shared resource pod volumes in various ways:

• The refreshResource attribute in the volumeAttributes properties.

• The refreshResources attribute in the Shared Resource CSI Driver configuration.

• The sharedSecret and sharedConfigMap attributes in the volumeAttributes properties.

The refreshResource attribute

The Shared Resource CSI Driver honors the refreshResource attribute in volumeAttributes properties of the volume. This attribute controls whether updates to the contents of the underlying Secret or ConfigMap object are copied to the volume after the volume is initially provisioned as part of pod startup. The default value of refreshResource is true, which means that the contents are updated.

The refreshResources attribute

You can use a global switch to enable or disable refreshing of shared resources. This switch is the refreshResources attribute in the csi-driver-shared-resource-config config map for the Shared Resource CSI Driver, which you can find in the openshift-cluster-csi-drivers namespace. If you set this refreshResources attribute to false, none of the Secret or ConfigMap object-related content stored in the volume is updated after the initial provisioning of the volume.

Validation of volumeAttributes before provisioning a shared resource volume for a pod

In the volumeAttributes of a single volume, you must set either a sharedSecret or a sharedConfigMap attribute to the value of a SharedSecret or a SharedConfigMap CS instance. Otherwise, when the volume is provisioned during pod startup, a validation checks the volumeAttributes of that volume and returns an error to the kubelet under the following conditions:

• Both sharedSecret and sharedConfigMap attributes have specified values.

• Neither sharedSecret nor sharedConfigMap attributes have specified values.

• The value of the sharedSecret or sharedConfigMap attribute does not correspond to the name of a SharedSecret or SharedConfigMap CR instance on the cluster.

Integration between shared resources, Insights Operator, and OKD Builds

Integration between shared resources, Insights Operator, and OKD Builds makes using Red Hat subscriptions (RHEL entitlements) easier in OKD Builds.

Previously, in OKD 4.9.x and earlier, you manually imported your credentials and copied them to each project or namespace where you were running builds.

Now, in OKD 4.10 and later, OKD Builds can use Red Hat subscriptions (RHEL entitlements) by referencing shared resources and the simple content access feature provided by Insights Operator:

• The simple content access feature imports your subscription credentials to a well-known Secret object. See the links in the following “Additional resources” section.

• The cluster administrator creates a SharedSecret custom resource (CR) instance around that Secret object and grants permission to particular projects or namespaces. In particular, the cluster administrator gives the builder service account permission to use that SharedSecret CR instance.

• Builds that run within those projects or namespaces can mount a CSI Volume that references the SharedSecret CR instance and its entitled RHEL content.

Additional resources

• Importing simple content access certificates with Insights Operator

• Adding subscription entitlements as a build secret