“Hybrid Retrieval in RAG: Why BM25 + Vector Search Needs Reciprocal Rank Fusion”

𝗛𝗼𝘄 𝗱𝗼𝗲𝘀 𝗛𝘆𝗯𝗿𝗶𝗱 𝗥𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹 𝘄𝗼𝗿𝗸 𝗶𝗻 𝗥𝗔𝗚? BM25 says 12. Vector search says 0.86. So… which document is actually more relevant? In a typical RAG system, we often use one retrieval method to find relevant documents. Most commonly, that's Vector Search — which focuses on the semantic meaning of the query. 𝗕𝘂𝘁 𝘄𝗵𝗮𝘁 𝗶𝗳 𝘁𝗵𝗲 𝗲𝘅𝗮𝗰𝘁 𝗸𝗲𝘆𝘄𝗼𝗿𝗱 𝗺𝗮𝘁𝘁𝗲𝗿𝘀 𝘁𝗼𝗼? That's where Hybrid Retrieval comes in. 𝗛𝘆𝗯𝗿𝗶𝗱 𝗥𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹 combines keyword search (matching exact words) with semantic search (matching meaning), then merges the results to find the most relevant information. Instead of relying on just one retrieval method, it combines: 🔹 𝗩𝗲𝗰𝘁𝗼𝗿 𝗦𝗲𝗮𝗿𝗰𝗵 → Understands semantic meaning 🔹 𝗕𝗠𝟮𝟱 → Matches exact keywords So instead of choosing between meaning and keywords, 𝗛𝘆𝗯𝗿𝗶𝗱 𝗥𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹 𝘁𝗿𝗶𝗲𝘀 𝘁𝗼 𝘂𝘀𝗲 𝗯𝗼𝘁𝗵. But now we have a problem... BM25 gives one score 12, while Vector Search gives another score 0.86. So how do we combine them? That's where 𝗥𝗲𝗰𝗶𝗽𝗿𝗼𝗰𝗮𝗹 𝗥𝗮𝗻𝗸 𝗙𝘂𝘀𝗶𝗼𝗻 (𝗥𝗥𝗙) comes in. Reciprocal Rank Fusion (RRF) combines the ranked results from different retrievers 𝘄𝗶𝘁𝗵𝗼𝘂𝘁 𝗰𝗼𝗺𝗽𝗮𝗿𝗶𝗻𝗴 𝘁𝗵𝗲𝗶𝗿 𝗿𝗮𝘄 𝘀𝗰𝗼𝗿𝗲𝘀. Example: Keyword search: Doc A is #1, Doc B is #2, Doc C is #3 Vector search: Doc C is #1, Doc A is #2, Doc D is #3 Formula: Score = 1 / (60 + rank), added up across every search Doc A → 1/61 + 1/62 = 0.0325 Doc C → 1/63 + 1/61 = 0.0323 𝗔 𝘄𝗶𝗻𝘀, 𝗯𝗲𝗰𝗮𝘂𝘀𝗲 𝗶𝘁 𝗿𝗮𝗻𝗸𝗲𝗱 𝘄𝗲𝗹𝗹 𝗶𝗻 𝗯𝗼𝘁𝗵 𝗹𝗶𝘀𝘁𝘀. The idea: 𝗮 𝗱𝗼𝗰𝘂𝗺𝗲𝗻𝘁 𝘁𝗵𝗮𝘁 𝗸𝗲𝗲𝗽𝘀 𝘀𝗵𝗼𝘄𝗶𝗻𝗴 𝘂𝗽 𝗻𝗲𝗮𝗿 𝘁𝗵𝗲 𝘁𝗼𝗽 𝗶𝘀 𝗽𝗿𝗼𝗯𝗮𝗯𝗹𝘆 𝘁𝗵𝗲 𝗼𝗻𝗲 𝘆𝗼𝘂 𝘄𝗮𝗻𝘁. That's why RRF is widely used in hybrid search and RAG systems 𝗜𝗻 𝗽𝗿𝗼𝗱𝘂𝗰𝘁𝗶𝗼𝗻 𝗥𝗔𝗚, 𝗯𝗲𝘁𝘁𝗲𝗿 𝗿𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹 𝗼𝗳𝘁𝗲𝗻 𝗺𝗲𝗮𝗻𝘀 𝗯𝗲𝘁𝘁𝗲𝗿 𝗮𝗻𝘀𝘄𝗲𝗿𝘀 What retrieval strategy are you using in your RAG system — Vector, BM25, or Hybrid?

How Hybrid Retrieval Works in RAG

Most RAG systems start with vector search, which retrieves documents by semantic meaning. But exact keywords sometimes matter too: product codes, names, acronyms, and technical terms that embeddings can blur.

Hybrid retrieval runs two retrievers in parallel and merges their results:

• Vector search: matches meaning
• BM25: matches exact keywords

The scoring problem: BM25 might return a score like 12, while vector search returns a cosine similarity like 0.86. These scales are incompatible, so you can’t simply add them.

The fix: Reciprocal Rank Fusion (RRF). RRF ignores raw scores and uses only rank positions:

Score = Σ 1 / (k + rank), with k = 60 by default

Example:

• Keyword ranking: A (#1), B (#2), C (#3)
• Vector ranking: C (#1), A (#2), D (#3)
• Doc A = 1/61 + 1/62 ≈ 0.0325
• Doc C = 1/63 + 1/61 ≈ 0.0323

A edges out C because it ranked highly in both lists, though the margin is small.

Takeaway: A document that consistently appears near the top across retrievers is likely relevant. RRF is simple, needs no score normalization, and is supported natively in many search platforms. Many production systems also add a re-ranker afterward for extra precision.

Caveats: RRF discards score magnitude, k is tunable, and hybrid search adds complexity, so test it against your own data.