llm-d
What ChatGPT, Claude, Gemini & Grok actually say · August 2026
The verdict
llm-d appears in 1 AI-ranked category.
Best Kubernetes-native choice for large distributed LLM deployments, combining vLLM with inference-aware routing, disaggregated prefill/decode, KV-cache coordination, and Gateway API integration
Where llm-d falls short, per the models
- GPT It is still comparatively young and overengineered for teams serving a few models on one or two nodes
Poll history — On this board 1 of 2 polls since Aug 3 — off it in the latest
#7 → –
Top alternatives per the models: vLLM · SGLang · KServe · TensorRT-LLM
Watch llm-d
Boards re-poll weekly and the models change their minds. One short email only when llm-d's standing moves — a rank change, a rival overtaking, or new reasoning from the models. Nothing otherwise.
Embed your ranking badge
llm-d ranks #7 for best open-source llm inference servers for kubernetes by AI-model consensus. Put the badge in your README, docs or site — it updates automatically as the models re-rank.
[](https://modelsagree.com/best/best-open-source-llm-inference-servers-for-kubernetes?utm_source=badge&utm_medium=embed&utm_campaign=badge-llm-d)<a href="https://modelsagree.com/best/best-open-source-llm-inference-servers-for-kubernetes?utm_source=badge&utm_medium=embed&utm_campaign=badge-llm-d"><img src="https://modelsagree.com/badge/llm-d.svg" alt="llm-d — ranked #7 for Best open-source LLM inference servers for Kubernetes by AI models on ModelsAgree" height="28"></a>Rankings are computed from what the models answer, re-polled on demand · raw reasoning shown verbatim · methodology