CLM-8B: Stanford and NVIDIA's Faster, More Honest Answer to Jev
Here's the simple idea:
Normal LLMs write answers word by word. That's slow.
CLM looks at the situation, looks at the possible actions, and picks the best match. No writing. Just choosing.
What the numbers say:
✅ Up to 9x faster than Jev
✅ 16.5 ms per decision vs 149.8 ms for Jev (T-Rex game test)
✅ 13x faster when choosing from ~1,000 options
✅ New best scores on coding agent benchmarks: DeepSWE 81.6%, Terminal-Bench 2.1 87.6%
✅ Free to use — Apache 2.0 weights
But let's be fair 👇
Jev was still more accurate in some tests. On tool-calling (BFCL v4), Jev hit 99.2% vs CLM's 95.2%. So CLM wins on speed, not always on accuracy.
Why I find this exciting:
Only ~20M parameters per head are trained on top of a frozen Qwen3-8B. That means fine-tuning is cheap, and you can run it on your own servers.
For anyone building AI agents, this matters. Agents make hundreds of small choices. If each choice is 10x faster, the whole agent feels instant.
A bigger multimodal CLM-35B is coming in early October. I'll be watching.
Check - https://lnkd.in/ghgbecXS
This is real, very recent (released September 23, 2026), and extensively covered by outlets including VentureBeat — most of the specific numbers check out precisely. Two things stand out as unconfirmed, and one thing is worth adding to make the "let's be fair" section actually complete.
Confirmed: Stanford and NVIDIA researchers released CLM-8B (Contrastive Language Model), Apache 2.0, positioned explicitly as an open answer to TypeSafe's Jev. The mechanism is described consistently across every source: instead of generating tokens, it creates vector representations of the current state and the available actions, then selects the action that best matches the state — a retrieval/matching approach rather than generation, matching your "looks at the situation, looks at the possible actions, picks the best match" framing precisely.
"Up to 9x faster" and "16.5ms vs 149.8ms on T-Rex" — both confirmed exactly, down to the decimal, appearing identically worded across multiple independent outlets: on the T-Rex game benchmark, CLM-8B clocked 16.5 milliseconds per decision versus Jev's 149.8 ms.
"DeepSWE 81.6%, Terminal-Bench 2.1 87.6%" — confirmed exactly, and actually understates the win. These aren't just new state-of-the-art scores in the abstract — they're direct wins over Jev on the same benchmarks: CLM reached 81.6% on DeepSWE and 87.6% on Terminal-Bench 2.1, compared with 71.1% and 83.1% for Jev. Worth adding, since your post only shows the benchmark where Jev wins (tool-calling) — a fuller "let's be fair" picture would note CLM actually beats Jev outright on both coding benchmarks, not just matches it on speed.
"Jev hit 99.2% vs CLM's 95.2% on BFCL v4" — confirmed exactly. Also worth adding for balance: Jev won a second test too — it completed 30 of 30 WikiRacing tasks versus CLM's 26 of 30 — so there are two areas where Jev held the edge, not just one, if you want the fairness section to be fully complete.
"~20M parameters per head on a frozen Qwen3-8B" — the frozen-base architecture is confirmed, but I couldn't verify the specific 20M figure. Multiple sources confirm CLM-8B is built on a frozen Qwen3-8B base with only additional projection heads for state and action inputs trained — the training-cheap, self-hostable framing is accurate — but the exact parameter count per head wasn't something I found stated directly.
Two things I could not verify at all:
- "13x faster when choosing from ~1,000 options" — plausible given reporting that the largest speedups appeared when choosing from a large candidate set, but I found no source stating this specific multiple or option count.
- "A bigger multimodal CLM-35B is coming in early October" — I found no mention of this in any coverage. This may be based on information not yet public, or it may not be accurate — worth treating as unconfirmed rather than repeating as a settled fact.
One technical detail worth adding for accuracy, since it's the actual reason for the speedup in large-option scenarios: the key architectural distinction, according to the CLM team, is that Jev and similar models mainly cache the state representation, while CLM's dual-encoder design lets it precompute and cache both the state and the action representations — which is specifically why it pulls further ahead as the number of candidate options grows.