GLang vs vLLM: What RadixAttention Actually Buys You on Agent Workloads

SGLang isn't just another inference engine; it's a serving stack co-designed around how real LLM programs actually look — structured prompts, tool calls, and multi-turn agents. Its RadixAttention KV cache reuses prefixes across requests, which routinely delivers 1.5–5× throughput wins on agent workloads where vLLM and TGI leave the cache cold.
Most "fast LLM serving" posts treat inference as a numbers problem — bigger batches, more FLOPs, lower p99. That's the vLLM story, and it's a good one. But if you've watched a production trace from a tool-using agent or a RAG pipeline, you know the real bottleneck isn't arithmetic. It's structure. The same system prompt shipped on every request. The same retrieved document chunk. The same JSON schema enforced across thousands of calls. Standard inference engines see these as unrelated requests with no shared state; SGLang sees them as the same computation, repeated.
That's the through-line of this post. We'll dig into what SGLang actually is, why its frontend language and runtime belong together, how RadixAttention works under the hood, and what changes when you put it in front of real traffic next to (or instead of) vLLM.
SGLang in Production: How Structured Generation Makes LLM Serving Actually Fast
Read the full guide: https://lnkd.in/daEuDSmY

The architectural description of SGLang is accurate; the specific throughput number is where this gets soft.

What SGLang is, and RadixAttention's mechanism — confirmed accurate. SGLang did emerge from LMSYS (the Chatbot Arena team) with a stated focus on optimizing complex, multi-step LLM programs rather than simple request-response patterns, and RadixAttention automatically discovers and exploits KV cache reuse using a radix tree indexed at the token level — this matches the post's description of the frontend-language-plus-runtime co-design almost exactly.

"1.5–5x throughput wins on agent workloads" is on the optimistic end of what current, independent 2026 benchmarks actually show. The original SGLang paper's headline number — up to 6.4x on prefix-heavy synthetic workloads — is where a figure like "5x" traces back to. But real-world 2026 head-to-head benchmarks against vLLM consistently land much lower: a 29% throughput advantage on H100 running Llama 3.1 8B (16,200 vs 12,500 tok/s), and for conversational/agent workloads specifically, RadixAttention delivers 10-20% performance improvements over vLLM. One benchmark shows the gap narrows to just 3-5% at 70B parameters. So "1.5-5x" sits between the original paper's peak lab number and the more modest gains people actually see now — not fabricated, but not the typical result either.

One important piece of context missing from the post: vLLM isn't cache-cold anymore. It has added its own automatic prefix caching since SGLang popularized the idea — the real differentiator today is that RadixAttention automatically discovers shared prefixes via a radix tree, while vLLM's approach requires more specific configuration for certain patterns. That's a real difference in ergonomics/coverage, but "vLLM leaves the cache cold" overstates it — vLLM does cache, just less automatically.

TGI is framed as a current peer, which is stale. Hugging Face put TGI into maintenance mode in December 2025 and now points new deployments toward vLLM and SGLang instead. A 2026 comparison should probably note that, since TGI isn't really in active contention anymore.

One thing that actually cuts against the post's framing: at high concurrency, SGLang's Python-based routing can hit GIL contention, and vLLM's C++/CUDA routing can pull ahead on raw throughput in that regime — so "SGLang wins on agent workloads" is conditional on prefix-overlap ratio and concurrency level, not a universal result.