Taints and Tolerations

If users want to create nodes with large storage spaces and/or CPU resources for Longhorn only (to store replica data) and reject other general workloads, they can taint those nodes and add tolerations for Longhorn components. Then Longhorn can be deployed on those nodes.

Notice that the taint tolerations setting for one workload will not prevent it from being scheduled to the nodes that don’t contain the corresponding taints.

For more information about how taints and tolerations work, refer to the official Kubernetes documentation.

Setting up Taints and Tolerations

Longhorn system contains user deployed components (e.g, Manager, Driver Deployer, UI) and system managed components (e.g, Instance Manager, Engine Image, CSI Driver, etc.) You need to set tolerations for both types of components. See more details below.

Setting up Taints and Tolerations During installing Longhorn

  1. Set taint tolerations for user deployed components (Manager, UI, Driver Deployer)

    • If you install Longhorn by Rancher 2.5.x, you need to click Edit as YAML in Rancher UI and copy this values into the YAML:

      1.   longhornManager:
      2.     tolerations:
      3.     - key: "key"
      4.       operator: "Equal"
      5.       value: "value"
      6.       effect: "NoSchedule"
      7.   longhornDriver:
      8.     tolerations:
      9.     - key: "key"
      10.       operator: "Equal"
      11.       value: "value"
      12.       effect: "NoSchedule"
      13.   longhornUI:
      14.     tolerations:
      15.     - key: "key"
      16.       operator: "Equal"
      17.       value: "value"
      18.       effect: "NoSchedule"

    • If you install Longhorn by using kubectl to apply the deployment YAML, you need to modify the taint tolerations section for Longhorn Manager, Longhorn UI, and Longhorn Driver Deployer. Then apply the YAMl files.

    • If you install Longhorn using Helm, you can change the Helm values for longhornManager.tolerations, longhornUI.tolerations, longhornDriver.tolerations in the values.yaml file. Then install the chart

  2. Set taint tolerations for system-managed components (for example, Instance Manager, CSI Driver, and Engine images)

    Follow the Customize default settings to set taint tolerations by changing the value for the taint-toleratio default setting

    Note: Because of the limitation of Rancher 2.5.x, if you are using Rancher UI to install Longhorn, you need to click Edit As Yaml and add setting taintToleration to defaultSettings.

    For example:

    1. defaultSettings:
    2.   taintToleration: "key=value:NoSchedule"

Setting up Taints and Tolerations After Longhorn has been installed

Warning:

To ensure that your preferred toleration settings are immediately applied, stop all workloads and detach all Longhorn volumes before configuring the settings.

Since all Longhorn components will be restarted, the Longhorn system is unavailable temporarily.

When all Longhorn volumes are detached, the customized setting is immediately applied to the system-managed components. When one or more Longhorn volumes are still attached, the customized setting is applied to the Instance Manager only when no engines and replica instances are running. You are required to reconfigure the setting after detaching the remaining volumes. Alternatively, you can wait for the next setting synchronization, which will occur in an hour.

Don’t operate the Longhorn system while toleration settings are updated and Longhorn components are being restarted.

  1. Prepare

    To ensure that your preferred settings are immediately applied, stop all workloads and detach all Longhorn volumes before configuring the settings.

  2. Set taint tolerations for user deployed components (Manager, UI, Driver Deployer)

    • If you install Longhorn by Rancher 2.5.x, you need to click Edit as YAML in Rancher UI and copy this values into the YAML:

      1.   longhornManager:
      2.     tolerations:
      3.     - key: "key"
      4.       operator: "Equal"
      5.       value: "value"
      6.       effect: "NoSchedule"
      7.   longhornDriver:
      8.     tolerations:
      9.     - key: "key"
      10.       operator: "Equal"
      11.       value: "value"
      12.       effect: "NoSchedule"
      13.   longhornUI:
      14.     tolerations:
      15.     - key: "key"
      16.       operator: "Equal"
      17.       value: "value"
      18.       effect: "NoSchedule"

    • If you install Longhorn by using kubectl to apply the deployment YAML, you need to modify the taint tolerations section for Longhorn Manager, Longhorn UI, and Longhorn Driver Deployer. Then reapply the YAMl files.

    • If you install Longhorn using Helm, you can change the Helm values for longhornManager.tolerations, longhornUI.tolerations, longhornDriver.tolerations in the values.yaml file. Then do Helm upgrade the chart.

  3. Set taint tolerations for system managed components

    The taint toleration setting can be found at Longhorn UI under Setting > General > Kubernetes Taint Toleration.

History

Available since v0.6.0