https://srekubecraft.io/tags/scheduling/Recent content in Scheduling on SREKubeCraft | Nick NikolakakisHugoen-usTue, 22 Sep 2026 00:00:00 +0000https://srekubecraft.io/posts/kubernetes-gang-scheduling-workload-api/Tue, 22 Sep 2026 00:00:00 +0000https://srekubecraft.io/posts/kubernetes-gang-scheduling-workload-api/<p>You give a four-rank training job a cluster with six GPUs. The scheduler places three ranks, runs out of devices, and leaves the fourth Pending. A second four-rank job arrives and takes the other three. Now six GPUs are allocated, two pods are Pending forever, and zero jobs are training. Nothing is broken. Every pod that could be scheduled was scheduled. That is exactly the problem.</p> <p>This is partial-placement deadlock, and until recently Kubernetes had no answer for it in tree. You reached for Volcano, or Kueue, or the coscheduling plugin, and you accepted a second scheduler or a controller that queued work outside the API you already had.</p>