LLM vs Jev: Generating Text vs Answering Typed Questions in Parallel

LLM vs Jev
The LLM side is the familiar autoregressive setup. It generates one token, then the next, until the response is complete. Even if the final output is JSON, it is still produced as a sequence of tokens.
Jev works differently. You give it some state and typed questions, and it returns typed probabilistic answers rather than generating a string.
Those questions can be a Choice, a Score, or a Noul. Choice picks from a fixed set, Score rates the state along ordered levels and can return a position between them, and Noul returns the probability that a statement is true.
If several questions use the same state, Jev can evaluate them together, independently and in parallel.

This is a precise, accurate technical description — several phrases match TypeSafe's own documentation and cookbooks almost word for word. There's one nuance worth adding about what the "parallel" batching benefit actually buys you.

The LLM/autoregressive framing — accurate and standard. Correctly describes token-by-token generation, including the point that JSON output is still produced as a token sequence, not returned as a structured object directly.

"You give it some state and typed questions, it returns typed probabilistic answers rather than generating a string" — confirmed exactly, matching Jev's core design across every independent source and TypeSafe's own docs: one shared context, several independent questions, and application logic that combines the results.

The three primitives (Choice, Score, Noul) — confirmed precisely, including the more subtle details:

• Choice picks one option from a set you define (up to 255) and returns a probability for each option plus a confidence value.
• Score returns a weighted position on an ordered rubric, capped at up to 10 levels — matching your "rates the state along ordered levels and can return a position between them" almost exactly (it's an expected/probability-weighted value, so it can land between discrete levels, not just on them).
• Noul returns a calibrated probability P(true) in [0,1] — confirmed exactly. One detail worth adding: Noul carries no separate confidence field, unlike Choice and Score, so code that reads answer.confidence uniformly across all three question types will break on Nouls specifically.

"If several questions use the same state, Jev can evaluate them together, independently and in parallel" — confirmed exactly, down to the named pattern. TypeSafe calls this speculative fan-out, and every question in a request uses the same state, evaluated independently and in parallel, so one answer never becomes context for another.

One important nuance worth adding, because it changes what "parallel" actually buys you in practice. The headline benchmark cookbook shows batching 13 questions into one call is 12.2x cheaper and 10x faster than 13 separate sequential calls — but that comparison is against sequential calls specifically. TypeSafe's own cookbook notes: concurrent separate calls would close the latency gap, not the cost gap, and batching had no measurable effect on the answers themselves beyond ordinary sampling noise. In other words, most of the real win is on cost (you send a large shared state once instead of N times), not necessarily on latency if you'd have parallelized your separate calls anyway.

Worth flagging for context, since it's not in the post: TypeSafe hasn't disclosed the underlying architecture in detail, and its published benchmarks are vendor-run — both mentioned consistently across independent coverage, and worth knowing whenever citing Jev's performance claims specifically.