Designing a RAG Pipeline That Knows Which Document Is Actually Authoritative

GenAI Interviewer Question Series Interview: Two retrieved documents contain different answers to the same question. One is from last year, while the other was updated yesterday. Question: How would you design the retrieval and ranking pipeline to identify the authoritative and most recent information? Explanation: Metadata Filtering: Store document version, effective date, source, department, and authority level as metadata during ingestion. Retrieval: Use hybrid search to retrieve semantically relevant documents, then apply metadata filters where appropriate. Authority Ranking: Assign higher ranking to trusted sources, such as official policies, over drafts or user-generated documents. Recency Ranking: Boost documents with newer effective dates, not simply upload timestamps. Reranking: Use a cross-encoder/reranker combining relevance, authority, version, and freshness signals. Conflict Detection: If two high-confidence sources still conflict, trigger a conflict-resolution step rather than blindly selecting one. Generation: Instruct the LLM to prioritize the authoritative, latest effective document and provide citations. Evaluation: Measure retrieval accuracy, freshness, source correctness, and conflict-resolution accuracy. _________________________________________________________________________ Preparing for an ML or AI interview or Looking for Transition in AI? Check out the resource below, it covers key concepts with 1000 interview question Answers, Roadmap, Projects and practical topics to help you prepare with confidence. 🔗 Link: https://lnkd.in/dez6Ji7E

Confirmed Accurate

• Metadata filtering (version, effective date, source, department, authority level) is a standard, well-established RAG technique. Storing and filtering on structured metadata alongside embeddings is a common and well-documented pattern for improving retrieval precision beyond pure semantic similarity.
• Hybrid search (semantic + filtering) is a real, widely-used approach. Combining dense vector retrieval with metadata or keyword filters is standard practice in production RAG systems, consistent with earlier posts in this series referencing BM25 + vector + RRF-style hybrid approaches.
• Reranking with a cross-encoder is a legitimate, established technique. Cross-encoder rerankers (as opposed to bi-encoder retrieval) are commonly used as a second-stage step to refine an initial candidate set using richer relevance signals — this is accurate.
• Recency/freshness weighting based on effective date rather than ingestion timestamp is a real and important distinction. Upload/ingestion time and document effective date are genuinely different signals, and conflating them is a known failure mode in time-sensitive RAG systems; correctly flagging this distinction is accurate and technically sound.

Mostly Accurate

• "Authority Ranking: Assign higher ranking to trusted sources... over drafts or user-generated documents" — This is a sound general principle, but source-authority scoring isn't a standardized off-the-shelf technique with a single accepted implementation; it requires custom authority-scoring logic (e.g., manually curated trust tiers, source-type weighting) built per organization. The post presents it as a clean pipeline step without acknowledging this requires bespoke design work.
• "Use a cross-encoder/reranker combining relevance, authority, version, and freshness signals" — Cross-encoders are typically trained or tuned for semantic relevance; combining relevance with authority/version/freshness in a single reranker typically requires either a custom-trained model or a separate weighted scoring layer on top of the reranker's output. The post blends these together as if they're a single off-the-shelf step, which simplifies real implementation complexity.

Partially Accurate

• "Conflict Detection: If two high-confidence sources still conflict, trigger a conflict-resolution step" — This is a reasonable design goal, but it's described as though it's a standard, drop-in pipeline component. In practice, automated conflict detection between semantically similar but factually different documents is a nontrivial NLP problem (requires claim extraction/comparison, not just retrieval scoring), and the post doesn't specify how the "conflict-resolution step" would actually work (human-in-the-loop review, majority voting across sources, LLM-based reconciliation, escalation, etc.) — the step is named but not substantively explained.

Not Fully Verified

• Whether this exact pipeline structure reflects a real, tested production system, or is a generalized "ideal answer" template for interview prep — The post is framed as an interview question/model answer, not a case study, so it should be read as a reasonable conceptual framework rather than a proven, benchmarked architecture. Its practicality (latency cost of multi-stage reranking plus conflict detection at query time) isn't discussed or validated.
• The linked resource's claim of "1000 interview question answers, roadmap, projects" — Cannot be verified without reviewing the actual content; no detail is given on quality, source, or currency of the material.

Opinion / Promotional Language

• The framing of the pipeline as the correct/complete answer to a "GenAI Interviewer" question is itself a stylistic and pedagogical choice — reasonable for interview prep content, but it presents one plausible design as though it's the definitive answer, when multiple valid architectures could address the same problem (e.g., simpler approaches using only recency + authority weighting without full conflict detection, depending on system requirements).
• "Looking for Transition in AI? Check out the resource below... to help you prepare with confidence" — standard promotional call to action for the linked resource.

Missing Context

• No discussion of latency/cost trade-offs. Adding metadata filtering, cross-encoder reranking, and a separate conflict-detection step all add computational cost and latency to the retrieval pipeline — relevant for any real system design discussion, and a strong interview answer would typically address this trade-off explicitly.
• No mention of how metadata (especially "authority level") is actually determined or maintained over time — this requires ongoing curation/governance, which is a nontrivial operational burden not addressed in the pipeline description.
• No mention of edge cases, such as what happens when effective-date metadata is missing, inconsistent, or manually falsified/incorrect at the source.
• The "Evaluation" step lists metrics (retrieval accuracy, freshness, source correctness, conflict-resolution accuracy) without specifying how any of them would actually be measured — e.g., "conflict-resolution accuracy" implies a labeled ground-truth dataset of conflicting-document scenarios, which is expensive to construct and not trivial to obtain.