https://srekubecraft.io/tags/vllm/Recent content in Vllm on SREKubeCraft | Nick NikolakakisHugoen-usSat, 15 Aug 2026 00:00:00 +0000https://srekubecraft.io/posts/llm-d-distributed-inference/Sat, 15 Aug 2026 00:00:00 +0000https://srekubecraft.io/posts/llm-d-distributed-inference/<p>Three months ago I <a href="https://srekubecraft.io/posts/kserve/">wrote about KServe</a> and how the <code>InferenceService</code> CRD had become the closest thing cloud-native has to a standard for putting a trained model behind an API. That post ended on a deliberate cliffhanger: KServe gives you a great single-model serving primitive, but it does not solve GPU sharing, fractional scheduling, or how you load-balance inference traffic across many replicas of a large model. I pointed at Volcano and Kueue and moved on.</p>