Poseidon-Firmament - An alternate scheduler

Current release of Poseidon-Firmament scheduler is an alpha release.

Poseidon-Firmament scheduler is an alternate scheduler that can be deployed alongside the default Kubernetes scheduler.

Introduction

Poseidon is a service that acts as the integration glue for the Firmament scheduler with Kubernetes. Poseidon-Firmament scheduler augments the current Kubernetes scheduling capabilities. It incorporates novel flow network graph based scheduling capabilities alongside the default Kubernetes Scheduler. Firmament scheduler models workloads and clusters as flow networks and runs min-cost flow optimizations over these networks to make scheduling decisions.

It models the scheduling problem as a constraint-based optimization over a flow network graph. This is achieved by reducing scheduling to a min-cost max-flow optimization problem. The Poseidon-Firmament scheduler dynamically refines the workload placements.

Poseidon-Firmament scheduler runs alongside the default Kubernetes Scheduler as an alternate scheduler, so multiple schedulers run simultaneously.

Key Advantages

Flow graph scheduling based Poseidon-Firmament scheduler provides the following key advantages:

• Workloads (pods) are bulk scheduled to enable scheduling at massive scale..
• Based on the extensive performance test results, Poseidon-Firmament scales much better than the Kubernetes default scheduler as the number of nodes increase in a cluster. This is due to the fact that Poseidon-Firmament is able to amortize more and more work across workloads.

• Poseidon-Firmament Scheduler outperforms the Kubernetes default scheduler by a wide margin when it comes to throughput performance numbers for scenarios where compute resource requirements are somewhat uniform across jobs (Replicasets/Deployments/Jobs). Poseidon-Firmament scheduler end-to-end throughput performance numbers, including bind time, consistently get better as the number of nodes in a cluster increase. For example, for a 2,700 node cluster (shown in the graphs here), Poseidon-Firmament scheduler achieves a 7X or greater end-to-end throughput than the Kubernetes default scheduler, which includes bind time.

• Availability of complex rule constraints.

• Scheduling in Poseidon-Firmament is dynamic; it keeps cluster resources in a global optimal state during every scheduling run.

• Highly efficient resource utilizations.

Poseidon-Firmament Scheduler - How it works

As part of the Kubernetes multiple schedulers support, each new pod is typically scheduled by the default scheduler. Kubernetes can be instructed to use another scheduler by specifying the name of another custom scheduler (“poseidon” in our case) in the schedulerName field of the PodSpec at the time of pod creation. In this case, the default scheduler will ignore that Pod and allow Poseidon scheduler to schedule the Pod on a relevant node.

apiVersion: v1
kind: Pod
...
spec:
    schedulerName: poseidon

Note: For details about the design of this project see the design document.

Possible Use Case Scenarios - When to use it

As mentioned earlier, Poseidon-Firmament scheduler enables an extremely high throughput scheduling environment at scale due to its bulk scheduling approach versus Kubernetes pod-at-a-time approach. In our extensive tests, we have observed substantial throughput benefits as long as resource requirements (CPU/Memory) for incoming Pods are uniform across jobs (Replicasets/Deployments/Jobs), mainly due to efficient amortization of work across jobs.

Although, Poseidon-Firmament scheduler is capable of scheduling various types of workloads, such as service, batch, etc., the following are a few use cases where it excels the most:

• For “Big Data/AI” jobs consisting of large number of tasks, throughput benefits are tremendous.
• Service or batch jobs where workload resource requirements are uniform across jobs (Replicasets/Deployments/Jobs).

Current Project Stage

• Alpha Release - Incubation repo. at https://github.com/kubernetes-sigs/poseidon.
• Currently, Poseidon-Firmament scheduler does not provide support for high availability, our implementation assumes that the scheduler cannot fail. The design document describes possible ways to enable high availability, but we leave this to future work.
• We are not aware of any production deployment of Poseidon-Firmament scheduler at this time.
• Poseidon-Firmament is supported from Kubernetes release 1.6 and works with all subsequent releases.
• Release process for Poseidon and Firmament repos are in lock step. The current Poseidon release can be found here and the corresponding Firmament release can be found here.

Features Comparison Matrix

Installation

For in-cluster installation of Poseidon, please start at the Installation instructions.

Development

For developers, please refer to the Developer Setup instructions.

Latest Throughput Performance Testing Results

Pod-by-pod schedulers, such as the Kubernetes default scheduler, typically process one pod at a time. These schedulers have the following crucial drawbacks:

• The scheduler commits to a pod placement early and restricts the choices for other pods that wait to be placed.
• There is limited opportunities for amortizing work across pods because they are considered for placement individually. These downsides of pod-by-pod schedulers are addressed by batching or bulk scheduling in Poseidon-Firmament scheduler. Processing several pods in a batch allows the scheduler to jointly consider their placement, and thus to find the best trade-off for the whole batch instead of one pod. At the same time it amortizes work across pods resulting in much higher throughput.

Note: Please refer to the latest benchmark results for detailed throughput performance comparison test results between Poseidon-Firmament scheduler and the Kubernetes default scheduler.

Feedback

Was this page helpful?

Thanks for the feedback. If you have a specific, answerable question about how to use Kubernetes, ask it onStack Overflow.Open an issue in the GitHub repo if you want toreport a problemorsuggest an improvement.