Longhorn PVC Ownership and Permission

Kubernetes supports the 2 volume modes for PVC: Filesystem and Block. When a pod defines the security context and requests a Longhorn PVC, Kubernetes will handle the ownership and permission modification for the PVC differently based on the volume mode.

Longhorn PVC with Filesystem Volume Mode

Because the Longhorn CSI driver csiDriver.spec.fsGroupPolicy is set to ReadWriteOnceWithFSType, the Kubelet attempts to change the ownership and permission of a Longhorn PVC in the following manner:

  1. Check pod.spec.securityContext.fsGroup.
  • If non-empty, continue to the next step.
  • If empty, the Kubelet doesn’t attempt to change the ownership and permission for the volume.
  1. Check fsType of the PV and accessModes of the PVC.
  • If the PV’s fsType is defined and the PVC’s accessModes list contains ReadWriteOnly, continue to the next step.
  • Otherwise, the Kubelet doesn’t attempt to change the ownership and permission for the volume.
  1. Check pod.spec.securityContext.fsGroupChangePolicy.
  • If the pod.spec.securityContext.fsGroupChangePolicy is set to always or empty, the kubelet performs the following actions:
    • Ensures that all processes of the containers inside the pod are part of the supplementary group id pod.spec.securityContext.fsGroup
    • Ensures that any new files created in the volume will be in group id pod.spec.securityContext.fsGroup
    • Recursively changes permission and ownership of the volume to have the same group id as pod.spec.securityContext.fsGroup every time the volume is mounted
  • If the pod.spec.securityContext.fsGroupChangePolicy is set to OnRootMismatch:
    • If the root of the volume already has the correct permissions (i.e., belongs to the group id as pod.spec.securityContext.fsGroup) , the recursive permission and ownership change will be skipped.
    • Otherwise, Kubelet recursively changes permission and ownership of the volume to have the same group id as pod.spec.securityContext.fsGroup

For more information, see:

Longhorn PVC with Block Volume Mode

For PVC with Block volume mode, Kubelet never attempts to change the permission and ownership of the block device when making it available inside the container. You must set the correct group ID in the pod.spec.securityContext for the pod to be able to read and write to the block device or run the container as root.

By default, Longhorn puts the block device into group id 6, which is typically associated with the “disk” group. Therefore, pods that use Longhorn PVC with Block volume mode must either set the group id 6 in the pod.spec.securityContext, or run as root. For example:

  1. Pod that sets the group id 6 in the pod.spec.securityContext

    1. apiVersion: v1
    2. kind: PersistentVolumeClaim
    3. metadata:
    4.   name: longhorn-block-vol
    5. spec:
    6.   accessModes:
    7.     - ReadWriteOnce
    8.   volumeMode: Block
    9.   storageClassName: longhorn
    10.   resources:
    11.     requests:
    12.       storage: 2Gi
    13. ---
    14. apiVersion: v1
    15. kind: Pod
    16. metadata:
    17.   name: block-volume-test
    18.   namespace: default
    19. spec:
    20.   securityContext:
    21.     runAsGroup: 1000
    22.     runAsNonRoot: true
    23.     runAsUser: 1000
    24.     supplementalGroups:
    25.     - 6
    26.   containers:
    27.     - name: block-volume-test
    28.       image: ubuntu:20.04
    29.       command: ["sleep", "360000"]
    30.       imagePullPolicy: IfNotPresent
    31.       volumeDevices:
    32.         - devicePath: /dev/longhorn/testblk
    33.           name: block-vol
    34.   volumes:
    35.     - name: block-vol
    36.       persistentVolumeClaim:
    37.         claimName: longhorn-block-vol

  2. Pod that runs as root

    1. apiVersion: v1
    2. kind: PersistentVolumeClaim
    3. metadata:
    4.   name: longhorn-block-vol
    5. spec:
    6.   accessModes:
    7.     - ReadWriteOnce
    8.   volumeMode: Block
    9.   storageClassName: longhorn
    10.   resources:
    11.     requests:
    12.       storage: 2Gi
    13. ---
    14. apiVersion: v1
    15. kind: Pod
    16. metadata:
    17.   name: block-volume-test
    18.   namespace: default
    19. spec:
    20.   containers:
    21.     - name: block-volume-test
    22.       image: ubuntu:20.04
    23.       command: ["sleep", "360000"]
    24.       imagePullPolicy: IfNotPresent
    25.       volumeDevices:
    26.         - devicePath: /dev/longhorn/testblk
    27.           name: block-vol
    28.   volumes:
    29.     - name: block-vol
    30.       persistentVolumeClaim:
    31.         claimName: longhorn-block-vol