Scheduling pods using a secondary scheduler

You can run a custom secondary scheduler in OKD by installing the Secondary Scheduler Operator, deploying the secondary scheduler, and setting the secondary scheduler in the pod definition.

Installing the Secondary Scheduler Operator

You can use the web console to install the Secondary Scheduler Operator for Red Hat OpenShift.

Prerequisites

  • You have access to the cluster with cluster-admin privileges.

  • You have access to the OKD web console.

Procedure

  1. Log in to the OKD web console.

  2. Create the required namespace for the Secondary Scheduler Operator for Red Hat OpenShift.

    1. Navigate to Administration → Namespaces and click Create Namespace.

    2. Enter openshift-secondary-scheduler-operator in the Name field and click Create.

  3. Install the Secondary Scheduler Operator for Red Hat OpenShift.

    1. Navigate to Operators → OperatorHub.

    2. Enter Secondary Scheduler Operator for Red Hat OpenShift into the filter box.

    3. Select the Secondary Scheduler Operator for Red Hat OpenShift and click Install.

    4. On the Install Operator page:

      1. The Update channel is set to stable, which installs the latest stable release of the Secondary Scheduler Operator for Red Hat OpenShift.

      2. Select A specific namespace on the cluster and select openshift-secondary-scheduler-operator from the drop-down menu.

      3. Select an Update approval strategy.

        • The Automatic strategy allows Operator Lifecycle Manager (OLM) to automatically update the Operator when a new version is available.

        • The Manual strategy requires a user with appropriate credentials to approve the Operator update.

      4. Click Install.

Verification

  1. Navigate to Operators → Installed Operators.

  2. Verify that Secondary Scheduler Operator for Red Hat OpenShift is listed with a Status of Succeeded.

Deploying a secondary scheduler

After you have installed the Secondary Scheduler Operator, you can deploy a secondary scheduler.

Prerequisities

  • You have access to the cluster with cluster-admin privileges.

  • You have access to the OKD web console.

  • The Secondary Scheduler Operator for Red Hat OpenShift is installed.

Procedure

  1. Log in to the OKD web console.

  2. Create config map to hold the configuration for the secondary scheduler.

    1. Navigate to Workloads → ConfigMaps.

    2. Click Create ConfigMap.

    3. In the YAML editor, enter the config map definition that contains the necessary KubeSchedulerConfiguration configuration. For example:

      1. apiVersion: v1
      2. kind: ConfigMap
      3. metadata:
      4.   name: "secondary-scheduler-config"                  (1)
      5.   namespace: "openshift-secondary-scheduler-operator" (2)
      6. data:
      7.   "config.yaml": |
      8.     apiVersion: kubescheduler.config.k8s.io/v1beta3
      9.     kind: KubeSchedulerConfiguration                  (3)
      10.     leaderElection:
      11.       leaderElect: false
      12.     profiles:
      13.       - schedulerName: secondary-scheduler            (4)
      14.         plugins:                                      (5)
      15.           score:
      16.             disabled:
      17.               - name: NodeResourcesBalancedAllocation
      18.               - name: NodeResourcesLeastAllocated
      1The name of the config map. This is used in the Scheduler Config field when creating the SecondaryScheduler CR.
      2The config map must be created in the openshift-secondary-scheduler-operator namespace.
      3The KubeSchedulerConfiguration resource for the secondary scheduler. For more information, see KubeSchedulerConfiguration in the Kubernetes API documentation.
      4The name of the secondary scheduler. Pods that set their spec.schedulerName field to this value are scheduled with this secondary scheduler.
      5The plug-ins to enable or disable for the secondary scheduler. For a list default scheduling plug-ins, see Scheduling plugins in the Kubernetes documentation.

    4. Click Create.

  3. Create the SecondaryScheduler CR:

    1. Navigate to Operators → Installed Operators.

    2. Select Secondary Scheduler Operator for Red Hat OpenShift.

    3. Select the Secondary Scheduler tab and click Create SecondaryScheduler.

    4. The Name field defaults to cluster; do not change this name.

    5. The Scheduler Config field defaults to secondary-scheduler-config. Ensure that this value matches the name of the config map created earlier in this procedure.

    6. In the Scheduler Image field, enter the image name for your custom scheduler.

      Red Hat does not directly support the functionality of your custom secondary scheduler.

    7. Click Create.

Scheduling a pod using the secondary scheduler

To schedule a pod using the secondary scheduler, set the schedulerName field in the pod definition.

Prerequisities

  • You have access to the cluster with cluster-admin privileges.

  • You have access to the OKD web console.

  • The Secondary Scheduler Operator for Red Hat OpenShift is installed.

  • A secondary scheduler is configured.

Procedure

  1. Log in to the OKD web console.

  2. Navigate to Workloads → Pods.

  3. Click Create Pod.

  4. In the YAML editor, enter the desired pod configuration and add the schedulerName field:

    1. apiVersion: v1
    2. kind: Pod
    3. metadata:
    4.   name: nginx
    5.   namespace: default
    6. spec:
    7.   containers:
    8.     - name: nginx
    9.       image: nginx:1.14.2
    10.       ports:
    11.         - containerPort: 80
    12.   schedulerName: secondary-scheduler (1)
    1The schedulerName field must match the name that is defined in the config map when you configured the secondary scheduler.

  5. Click Create.

Verification

  1. Log in to the OpenShift CLI.

  2. Describe the pod using the following command:

    1. $ oc describe pod nginx -n default

    Example output

    1. Name:         nginx
    2. Namespace:    default
    3. Priority:     0
    4. Node:         ci-ln-t0w4r1k-72292-xkqs4-worker-b-xqkxp/10.0.128.3
    5. ...
    6. Events:
    7.   Type    Reason          Age   From                 Message
    8.   ----    ------          ----  ----                 -------
    9.   Normal  Scheduled       12s   secondary-scheduler  Successfully assigned default/nginx to ci-ln-t0w4r1k-72292-xkqs4-worker-b-xqkxp
    10. ...

  3. In the events table, find the event with a message similar to Successfully assigned <namespace>/<pod_name> to <node_name>.

  4. In the “From” column, verify that the event was generated from the secondary scheduler and not the default scheduler.

    You can also check the secondary-scheduler-* pod logs in the openshift-secondary-scheduler-namespace to verify that the pod was scheduled by the secondary scheduler.