K
pending
The Silent Failure Mode: Why Partial Retrieval Beats Hallucination as AI's Real Problem
Grounded / Real
Inflated / Uruttu
Original Content
Stop optimizing tokens. Index your context instead!
Retrieval quality is the foundation of context engineering. Karpathy described it best: the heavy cognitive work should happen at ingestion, not at query time. When knowledge is properly structured before retrieval, the model's job becomes reasoning, not sorting.
Most AI systems focus on compressing what the model sees. The more important problem is what gets retrieved before the model sees anything.
Token efficiency starts at retrieval. When context is properly indexed and prepared, the model spends its tokens on reasoning. When retrieval is weak, the model spends those same tokens sorting through noise and filling gaps from its own weights.
That's the silent failure mode. Retrieval returns topically correct but incomplete context. The model completes the gaps from parametric knowledge and streams it out the same way as grounded content. No signal in the output tells you which parts came from retrieved context and which came from weights. The answer looks confident. It just isn't complete.
The common assumption is that hallucination is the main failure mode. It's not. Models handle off-domain questions reasonably well now. If nothing in the retrieved context looks relevant, there's no material to build an answer on. The harder failure is partial coverage. The right document was retrieved. But not the full picture.
Coverage gaps don't produce error messages. They produce confident answers with pieces missing.
This gets worse when sources stay isolated. The same person might appear across multiple tools and systems. If those sources are indexed separately, the model has to figure out they refer to the same entity on its own. That's work that should happen before the model starts reasoning.
Glean's system of context is built around this problem:
• Unified index across all connected applications, not each source kept separate
• Specialized indexes for different types of information: company data, code, experts, profiles, tools, and calendars
• Multiple retrieval methods - semantic when meaning matters, lexical when exact terms matter, structured when fields and relationships need to stay intact
• Enterprise Graph that maps relationships across people, teams, customers, and projects so relevance reflects how the company actually works
• Memory that carries context forward across sessions and tasks
• Tools that let the model act on what it finds
The gap between finding information and understanding it is where most AI systems fall short.
I've shared the link in the replies!
Retrieval quality is the foundation of context engineering. Karpathy described it best: the heavy cognitive work should happen at ingestion, not at query time. When knowledge is properly structured before retrieval, the model's job becomes reasoning, not sorting.
Most AI systems focus on compressing what the model sees. The more important problem is what gets retrieved before the model sees anything.
Token efficiency starts at retrieval. When context is properly indexed and prepared, the model spends its tokens on reasoning. When retrieval is weak, the model spends those same tokens sorting through noise and filling gaps from its own weights.
That's the silent failure mode. Retrieval returns topically correct but incomplete context. The model completes the gaps from parametric knowledge and streams it out the same way as grounded content. No signal in the output tells you which parts came from retrieved context and which came from weights. The answer looks confident. It just isn't complete.
The common assumption is that hallucination is the main failure mode. It's not. Models handle off-domain questions reasonably well now. If nothing in the retrieved context looks relevant, there's no material to build an answer on. The harder failure is partial coverage. The right document was retrieved. But not the full picture.
Coverage gaps don't produce error messages. They produce confident answers with pieces missing.
This gets worse when sources stay isolated. The same person might appear across multiple tools and systems. If those sources are indexed separately, the model has to figure out they refer to the same entity on its own. That's work that should happen before the model starts reasoning.
Glean's system of context is built around this problem:
• Unified index across all connected applications, not each source kept separate
• Specialized indexes for different types of information: company data, code, experts, profiles, tools, and calendars
• Multiple retrieval methods - semantic when meaning matters, lexical when exact terms matter, structured when fields and relationships need to stay intact
• Enterprise Graph that maps relationships across people, teams, customers, and projects so relevance reflects how the company actually works
• Memory that carries context forward across sessions and tasks
• Tools that let the model act on what it finds
The gap between finding information and understanding it is where most AI systems fall short.
I've shared the link in the replies!
Validated Content
Retrieval quality, not token compression, is the real foundation of context engineering — the heavy cognitive work should happen when knowledge is indexed, not when it's queried. When retrieval is weak, models spend tokens sorting through noise and filling gaps from their own trained knowledge instead of reasoning, and there's no signal in the output distinguishing retrieved content from generated content. This creates a failure mode distinct from hallucination: partial coverage, where the right document is retrieved but the full picture isn't, producing confident but incomplete answers. This problem worsens when data sources are indexed separately, since the model then has to infer on its own that references across tools point to the same entity. Glean's system addresses this through a unified index across connected applications, specialized indexes for different data types (company data, code, experts, profiles, tools, calendars), multiple retrieval methods (semantic, lexical, structured), an Enterprise Graph mapping relationships across people and teams, memory that persists across sessions, and tools that let the model act on retrieved information.