Node Device Manager User Guide

This section provides the operations need to performed by the Admin for configuring NDM.

Admin operations

Include filters

Exclude filters

Create blockdevice CRs for partitioned disks

Admin Operations

Include filters

To include only selected disks for provisioning, update the operator YAML file with the required blockdevices under NDM configuration section so that only these blockdevice will be taken for the creation of blockdevice CR. Add the blockdevice path in the following configuration for specifying particular disks. This configuration is added in openebs-ndm-config under Configmap in openebs-operator.yaml.

This change must be done in the openebs-operator.yaml file that you have downloaded before OpenEBS installation.

  1. filterconfigs:
  2. - key: path-filter
  3.   name: path filter
  4.   state: true
  5.   include: "/dev/sda"
  6.   exclude: ""

When the above configuration is used, only the block device /dev/sda will be used for creating the block device custom resource. All other disks will be excluded.

Exclude filters

NDM do some filtering on the disks to exclude, for example boot disk. By default, NDM excludes the following device path while creating block device CR. This configuration is added in openebs-ndm-config under Configmap in openebs-operator.yaml.

  1. /dev/loop - loop devices.
  2. /dev/fd - file descriptors.
  3. /dev/sr - CD-ROM devices.
  4. /dev/ram - ramdisks.
  5. /dev/dm -lvm.
  6. /dev/md -multiple device ( software RAID devices).

The following is the snippet of NDM configuration file from openebs operator YAML which excludes the provided disks/paths.

  1. filterconfigs:
  2.   - key: os-disk-exclude-filter
  3.     name: os disk exclude filter
  4.     state: true
  5.     exclude: "/,/etc/hosts,/boot"
  6.   - key: vendor-filter
  7.     name: vendor filter
  8.     state: true
  9.     include: ""
  10.     exclude: "CLOUDBYT,OpenEBS"
  11.   - key: path-filter
  12.     name: path filter
  13.     state: true
  14.     include: ""
  15.     exclude: "loop,/dev/fd0,/dev/sr0,/dev/ram,/dev/dm-,/dev/md"

It is also possible to customize by adding more disk types associated with your nodes. For example, used disks, unwanted disks and so on. This change must be done in the ‘openebs-operator.yaml’ file that you have downloaded before OpenEBS installation.

  1. filterconfigs:
  2.   - key: path-filter
  3.     name: path filter
  4.     state: true
  5.     include: ""
  6.     exclude: "loop,/dev/fd0,/dev/sr0,/dev/ram,/dev/dm-,/dev/md"

Note: It is recommended to use OpenEBS provisioner alone in the cluster. If you are using other storage provider provisioner like gce-pd along with OpenEBS, use exclude filters to avoid those disks from being consumed by OpenEBS. For example, if you are using the standard storage class in GKE with storage provisioner as kubernetes.io/gce-pd, and when it creates a PVC, a GPD is attached to the node. This GPD will be detected by NDM and it may be used by OpenEBS for provisioning volume. To avoid this scenario, it is recommended to put the associated device path created on the node in the exclude field under path-filter. If GPD is attached as /dev/sdc , then add /dev/sdc in the above mentioned field.

Example snippet:

In the downloaded openebs-operator.yaml, find openebs-ndm-config configmap and update the values for path-filter and any other filters if required.

  1. ---
  2. # This is the node-disk-manager related config.
  3. # It can be used to customize the disks probes and filters
  4. apiVersion: v1
  5. kind: ConfigMap
  6. metadata:
  7.  name: openebs-ndm-config
  8.  namespace: openebs
  9. data:
  10.  # udev-probe is default or primary probe which should be enabled to run ndm
  11.  # filterconfigs contails configs of filters - in their form fo include
  12.  # and exclude comma separated strings
  13.  node-disk-manager.config: |
  14.    probeconfigs:
  15.      - key: udev-probe
  16.        name: udev probe
  17.        state: true
  18.      - key: seachest-probe
  19.        name: seachest probe
  20.        state: false
  21.      - key: smart-probe
  22.        name: smart probe
  23.        state: true
  24.    filterconfigs:
  25.      - key: os-disk-exclude-filter
  26.        name: os disk exclude filter
  27.        state: true
  28.        exclude: "/,/etc/hosts,/boot"
  29.      - key: vendor-filter
  30.        name: vendor filter
  31.        state: true
  32.        include: ""
  33.        exclude: "CLOUDBYT,OpenEBS"
  34.      - key: path-filter
  35.        name: path filter
  36.        state: true
  37.        include: ""
  38.        exclude: "loop,/dev/fd0,/dev/sr0,/dev/ram,/dev/dm-,/dev/md,/dev/sdc"
  39. ---

Create blockdevice CRs for partitioned disks

Currently, NDM is not selecting partitioned disks for creating device resource. But, you can create block device resource for the partitioned disks manually. The following are the steps for the creation of block device resource.

  1. Create the sample block device CR YAML using the following spec. Following is the sample block device CR YAML.

    1. apiVersion: openebs.io/v1alpha1
    2. kind: BlockDevice
    3. metadata:
    4.   name: example-blockdevice
    5.   labels:
    6.     kubernetes.io/hostname: <host name in which disk/blockdevice is attached> # like gke-openebs-user-default-pool-044afcb8-bmc0
    7.     ndm.io/managed: "false" # for manual disk creation put false
    8.     ndm.io/blockdevice-type: blockdevice
    9. status:
    10.   claimState: Unclaimed
    11.   state: Active
    12. spec:
    13.   capacity:
    14.     logicalSectorSize: <logical sector size of lockdevice> # like 512
    15.     storage: <total capacity in bits> #like 53687091200
    16.   details:
    17.     firmwareRevision: <firmware revision>
    18.     model: <model name of blockdevice> # like PersistentDisk
    19.     serial: <serial no of disk> # like google-disk-2
    20.     compliance: <compliance of disk> #like "SPC-4"
    21.     vendor: <vendor of disk> #like Google
    22.   devlinks:
    23.   - kind: by-id
    24.     links:
    25.     - <link1> # like /dev/disk/by-id/scsi-0Google_PersistentDisk_disk-2
    26.     - <link2> # like /dev/disk/by-id/google-disk-2
    27.   - kind: by-path
    28.     links:
    29.     - <link1> # like /dev/disk/by-path/virtio-pci-0000:00:03.0-scsi-0:0:2:0 
    30.   Partitioned: Yes
    31.   path: <devpath> # like /dev/sdb1

  2. Modify the created block device CR sample YAML with the partition disk information. In the above block device CR sample spec, following fields must be filled before applying the YAML.

    • kubernetes.io/hostname
      • Hostname where the blockdevice is attached.
    • storage
    • links
      • This field should be filled for by-id and by-path. These details can be obtained from worker node by running the following command udevadm info -q property -n <device_path>
    • path
      • The value should be like /dev/sdb1 .

  3. Apply the modified YAML file to create the blockdevice CR for the provided partitioned device path.

  4. Repeat the same steps for each partitioned device and create blockdevice CR for each device.

  5. Verify if the blockdevice is created by running the following kubectl get blockdevice -n openebs command.

Note: If you are creating a block device CR for a partitioned device path, then you must add the corresponding disk under exclude filter so that NDM will not select the particular disk for BD creation. For example, /dev/sdb have 2 partitions, say /dev/sdb1 and /dev/sdb2. If block device CR is creating for /dev/sdb1 manually, then you must add /dev/sdb under exclude filter of NDM configuration. See here for customizing the exclude filter in NDM configuration.

See Also:

Understanding Node Disk Manager