Resource Bin Packing

In the scheduling-plugin NodeResourcesFit of kube-scheduler, there are two scoring strategies that support the bin packing of resources: MostAllocated and RequestedToCapacityRatio.

Enabling bin packing using MostAllocated strategy

The MostAllocated strategy scores the nodes based on the utilization of resources, favoring the ones with higher allocation. For each resource type, you can set a weight to modify its influence in the node score.

To set the MostAllocated strategy for the NodeResourcesFit plugin, use a scheduler configuration similar to the following:

  1. apiVersion: kubescheduler.config.k8s.io/v1
  2. kind: KubeSchedulerConfiguration
  3. profiles:
  4. - pluginConfig:
  5.   - args:
  6.       scoringStrategy:
  7.         resources:
  8.         - name: cpu
  9.           weight: 1
  10.         - name: memory
  11.           weight: 1
  12.         - name: intel.com/foo
  13.           weight: 3
  14.         - name: intel.com/bar
  15.           weight: 3
  16.         type: MostAllocated
  17.     name: NodeResourcesFit

To learn more about other parameters and their default configuration, see the API documentation for NodeResourcesFitArgs.

Enabling bin packing using RequestedToCapacityRatio

The RequestedToCapacityRatio strategy allows the users to specify the resources along with weights for each resource to score nodes based on the request to capacity ratio. This allows users to bin pack extended resources by using appropriate parameters to improve the utilization of scarce resources in large clusters. It favors nodes according to a configured function of the allocated resources. The behavior of the RequestedToCapacityRatio in the NodeResourcesFit score function can be controlled by the scoringStrategy field. Within the scoringStrategy field, you can configure two parameters: requestedToCapacityRatio and resources. The shape in the requestedToCapacityRatio parameter allows the user to tune the function as least requested or most requested based on utilization and score values. The resources parameter consists of name of the resource to be considered during scoring and weight specify the weight of each resource.

Below is an example configuration that sets the bin packing behavior for extended resources intel.com/foo and intel.com/bar using the requestedToCapacityRatio field.

  1. apiVersion: kubescheduler.config.k8s.io/v1
  2. kind: KubeSchedulerConfiguration
  3. profiles:
  4. - pluginConfig:
  5.   - args:
  6.       scoringStrategy:
  7.         resources:
  8.         - name: intel.com/foo
  9.           weight: 3
  10.         - name: intel.com/bar
  11.           weight: 3
  12.         requestedToCapacityRatio:
  13.           shape:
  14.           - utilization: 0
  15.             score: 0
  16.           - utilization: 100
  17.             score: 10
  18.         type: RequestedToCapacityRatio
  19.     name: NodeResourcesFit

Referencing the KubeSchedulerConfiguration file with the kube-scheduler flag --config=/path/to/config/file will pass the configuration to the scheduler.

To learn more about other parameters and their default configuration, see the API documentation for NodeResourcesFitArgs.

Tuning the score function

shape is used to specify the behavior of the RequestedToCapacityRatio function.

  1. shape:
  2.   - utilization: 0
  3.     score: 0
  4.   - utilization: 100
  5.     score: 10

The above arguments give the node a score of 0 if utilization is 0% and 10 for utilization 100%, thus enabling bin packing behavior. To enable least requested the score value must be reversed as follows.

  1. shape:
  2.   - utilization: 0
  3.     score: 10
  4.   - utilization: 100
  5.     score: 0

resources is an optional parameter which defaults to:

  1. resources:
  2.   - name: cpu
  3.     weight: 1
  4.   - name: memory
  5.     weight: 1

It can be used to add extended resources as follows:

  1. resources:
  2.   - name: intel.com/foo
  3.     weight: 5
  4.   - name: cpu
  5.     weight: 3
  6.   - name: memory
  7.     weight: 1

The weight parameter is optional and is set to 1 if not specified. Also, the weight cannot be set to a negative value.

Node scoring for capacity allocation

This section is intended for those who want to understand the internal details of this feature. Below is an example of how the node score is calculated for a given set of values.

Requested resources:

  1. intel.com/foo : 2
  2. memory: 256MB
  3. cpu: 2

Resource weights:

  1. intel.com/foo : 5
  2. memory: 1
  3. cpu: 3

FunctionShapePoint {{0, 0}, {100, 10}}

Node 1 spec:

  1. Available:
  2.   intel.com/foo: 4
  3.   memory: 1 GB
  4.   cpu: 8
  6. Used:
  7.   intel.com/foo: 1
  8.   memory: 256MB
  9.   cpu: 1

Node score:

  1. intel.com/foo  = resourceScoringFunction((2+1),4)
  2.                = (100 - ((4-3)*100/4)
  3.                = (100 - 25)
  4.                = 75                       # requested + used = 75% * available
  5.                = rawScoringFunction(75)
  6.                = 7                        # floor(75/10)
  8. memory         = resourceScoringFunction((256+256),1024)
  9.                = (100 -((1024-512)*100/1024))
  10.                = 50                       # requested + used = 50% * available
  11.                = rawScoringFunction(50)
  12.                = 5                        # floor(50/10)
  14. cpu            = resourceScoringFunction((2+1),8)
  15.                = (100 -((8-3)*100/8))
  16.                = 37.5                     # requested + used = 37.5% * available
  17.                = rawScoringFunction(37.5)
  18.                = 3                        # floor(37.5/10)
  20. NodeScore   =  ((7 * 5) + (5 * 1) + (3 * 3)) / (5 + 1 + 3)
  21.             =  5

Node 2 spec:

  1. Available:
  2.   intel.com/foo: 8
  3.   memory: 1GB
  4.   cpu: 8
  5. Used:
  6.   intel.com/foo: 2
  7.   memory: 512MB
  8.   cpu: 6

Node score:

  1. intel.com/foo  = resourceScoringFunction((2+2),8)
  2.                =  (100 - ((8-4)*100/8)
  3.                =  (100 - 50)
  4.                =  50
  5.                =  rawScoringFunction(50)
  6.                = 5
  8. memory         = resourceScoringFunction((256+512),1024)
  9.                = (100 -((1024-768)*100/1024))
  10.                = 75
  11.                = rawScoringFunction(75)
  12.                = 7
  14. cpu            = resourceScoringFunction((2+6),8)
  15.                = (100 -((8-8)*100/8))
  16.                = 100
  17.                = rawScoringFunction(100)
  18.                = 10
  20. NodeScore   =  ((5 * 5) + (7 * 1) + (10 * 3)) / (5 + 1 + 3)
  21.             =  7

What’s next