DM Explains · No. 008 · RAG
Hybrid multimodal retrieval: why more search paths can help or hurt
Combining keyword, vector and image search fixes what any single method misses — but only if you fuse them carefully, because one weak path can drag the whole answer down.
· 14 min read
Read me first
Combining keyword search, vector search and image search genuinely does find evidence that any single method misses, but only when the fusion step is tuned with care — one weak retrieval path can drag the whole answer down rather than help it.
Scientific paper search across scanned pages → 1 lexical, dense and image embeddings run in parallel → 2 routing avoids favouring text-only matches → 3 RRF fuses ranks (k=25) → 4 cross-encoder reranker isolates top 3 contexts → 5 k=5 pages retrieved beats single best page for complex queries → 6 answer grounded across pages
A team building question-answering over scientific papers kept getting confident wrong answers. The generation model wrote fluent paragraphs, but the evidence behind them was thin: a table scanned as an image, a caption sitting inside a diagram, a rare instrument name that no dense embedding had ever seen written that way. The dense vector index, tuned for meaning, walked straight past pages it never learned to "see". The keyword index never got a chance, because the page it needed was a picture, not text.
Get retrieval wrong in a system like this and nobody downstream can tell. The generation model still writes a tidy, well-formatted answer; it just built that answer on whatever evidence retrieval handed it, wrong or right. The fix that teams reach for is obvious: run more kinds of search at once. The harder truth is that doing this well is not obvious at all.
Everyday picture. Amma checks the exact index card labelled "biryani". Meera flips through the cookbook for anything that looks or reads similar even without that title. Dadi remembers a photo of the dish from an old magazine. The three compare notes and agree on the best match.
Where the picture breaks. In the house, a clearly wrong suggestion just gets ignored — nobody outvotes Amma's correct card because Dadi misremembered a photo. In hybrid retrieval, one weak method's poor answers can mathematically drag down the fused ranking, even when the other two methods were right.
Why it exists now
Retrieval-augmented generation systems have long defaulted to one retrieval method: usually dense vector search, because it captures paraphrase and meaning well. But real corpora are not made of clean, typed prose. Scientific papers mix free text, tables and figures; product catalogues mix names, codes and photos; email mixes structured metadata with body text. A single embedding space understands some of that and misses the rest.
What changed is that this stopped being a hunch and became measured fact. A VLDB 2026 study systematically tested hybrid architectures that integrate four retrieval paradigms — full text search, sparse vector search, dense vector search and tensor search — across 11 real-world datasets. Benchmarks built specifically for mixed-format documents, such as IRPAPERS in 2026, went further and tested text-based and image-based retrieval directly on scanned scientific paper pages. The result across this work is consistent: combining methods well beats any single method, but combining methods badly can beat nothing.
How it works
Parallel paths
No single method sees everything at once
What it is. A query is sent at the same time to a lexical index, a dense vector index, and — for mixed-format corpora — an image-based index, each returning its own ranked list of candidates.
Why it exists. Each method sees a different slice of relevance: lexical search catches exact terms and codes, dense vectors catch semantic similarity across paraphrase, and finer-grained multi-vector methods catch token-level alignment that a single embedding compresses away.
Where it breaks. Running every path on every query costs latency and compute even when one method alone would have been enough — there is no free lunch for breadth.
How that's handled. Systems tune which paths to run per query type rather than always firing all of them, and some newer approaches learn to orchestrate search paths programmatically instead of running everything by default.
In technical terms
In technical terms: the VLDB 2026 study evaluated hybrid search architectures integrating full text search, sparse vector search, dense vector search, and tensor search side by side across 11 real-world datasets.
Modality routing
Same-modality bias is a trap, not a feature
What it is. For corpora with text and images, retrieval keeps modality-specific indexes separate rather than forcing everything into one shared embedding space, then routes or merges queries according to what is needed.
Why it exists. Forcing text and images into a single aggregated embedding space creates what researchers call a "modality gap": retrieval starts favouring results that share the query's own modality rather than the item that is actually most relevant.
Where it breaks. Routing decisions can misfire on genuinely ambiguous queries that need both a text passage and an image at once, and there is no clean rule for detecting those cases automatically.
How that's handled. UniversalRAG addresses this with modality-aware routing, deliberately keeping retrieval paths distinct before any fusion happens rather than collapsing modalities early.
In technical terms
In technical terms: UniversalRAG proposes modality-aware routing specifically to counter the modality gap that appears when all modalities are aggregated into one embedding space.
Score fusion
Different scales cannot just be added together
What it is. The ranked lists from each path get merged — commonly with Reciprocal Rank Fusion, sometimes with a weighted alpha blend of vector and lexical scores — rather than by comparing raw relevance scores directly.
Why it exists. A BM25 keyword score and a cosine-similarity vector score do not live on the same scale, so adding them raw is meaningless; rank-based fusion sidesteps that by only caring about each result's position in its own list.
Where it breaks. Across the VLDB study's 11 datasets, hybrid architectures showed a "weakest link" effect: including one substantially weaker retrieval path in the fusion degraded overall accuracy rather than improving it. More paths is not automatically better search.
How that's handled. A multimodal scientific-document RAG pipeline tunes Reciprocal Rank Fusion with a constant of k=25 across its dense HNSW and sparse GIN result lists before reranking, rather than fusing every available signal blindly. HetaRAG instead uses a configurable alpha parameter to blend vector and keyword scores, so the weighting can be adjusted per corpus.
In technical terms
In technical terms: Reciprocal Rank Fusion sums the reciprocal of each item's rank across result lists; the multimodal scientific RAG pipeline tunes this with k=25 for dense HNSW plus sparse GIN retrieval, while HetaRAG instead blends vector and lexical scores through a configurable alpha parameter.
Reranking
Fusion is coarse; reranking looks closely
What it is. The fused shortlist is passed through a cross-encoder or tensor-based reranker that scores each candidate against the query directly, rather than relying on the coarse arithmetic of fusion.
Why it exists. Fusion only approximates relevance from rank positions; a reranker examines the actual text of query and candidate together, catching relevance that fusion's arithmetic could not see.
Where it breaks. Rerankers are only affordable on a small shortlist — running one across an entire corpus would be too slow and expensive — so a bad fusion step upstream still caps what reranking can recover.
How that's handled. The scientific-document RAG pipeline retrieves a batch of candidates per method before fusion, then reranks with a cross-encoder (ms-marco-MiniLM-L6-v2) down to the top 3 contexts. The VLDB study also flags tensor-based re-ranking fusion as a cheaper alternative that approximates full tensor search's semantic power at a fraction of its computational and memory cost.
In technical terms
In technical terms: the multimodal scientific RAG pipeline fuses candidate lists from dense and sparse retrieval, then reranks with a cross-encoder (ms-marco-MiniLM-L6-v2) to isolate the top 3 highest-scoring contexts.
Context depth
One perfect page is often not enough
What it is. A final decision on how many top-ranked results (k) to pass into the generation model's context window, separate from how those results were ranked.
Why it exists. Complex queries often need evidence spread across more than one page or chunk; handing the generation model only the single top result can starve it of context it needs to answer correctly.
Where it breaks. More context costs more tokens, and if irrelevant pages sneak into the top-k, they can dilute the generation model's attention rather than help it.
How that's handled. On the IRPAPERS benchmark, retrieving k=5 pages for complex queries beat retrieving a single best page. Text retrieval using hybrid text search reached 46% Recall@1, 78% Recall@5 and 91% Recall@20, while closed-source Cohere Embed v4 page image embeddings reached 58% Recall@1, 87% Recall@5 and 97% Recall@20, outperforming Voyage 3 Large text embeddings and all tested open-source models.
In technical terms
In technical terms: on IRPAPERS, hybrid text search achieved 46% Recall@1, 78% Recall@5, 91% Recall@20, while closed-source Cohere Embed v4 page image embeddings reached 58% Recall@1, 87% Recall@5, 97% Recall@20 — ahead of Voyage 3 Large text embeddings and all tested open-source models — and k=5 outperformed k=1 for complex queries.
Compared with what came before
| Single-method search | Hybrid multimodal retrieval | |
|---|---|---|
| What it catches | Whatever that one method is good at — exact terms, or semantic paraphrase, but not both | Lexical exactness, semantic meaning, and image content, fused together |
| Cost per query | One index lookup | Multiple index lookups plus a fusion step, often plus reranking |
| Failure mode | Predictable: misses whatever the method structurally cannot see | Can be worse than single-method if one path is weak and dilutes the fusion |
| Tuning burden | Low — one index, one scoring function | Higher — fusion constants, alpha weights, per-path recall need checking |
| Best fit | Homogeneous, mostly textual corpora near ceiling recall already | Mixed-format corpora: scanned pages, tables, figures, rare terms |
Hybrid multimodal retrieval earns its cost when the corpus itself is mixed and no single method is close to covering it; it is a needless tax when a corpus is homogeneous enough that one well-tuned method already sits near its recall ceiling.
Where it breaks
Adding more retrieval paths helps only when each path is worth including; a path that performs poorly on its own can pull the fused ranking down rather than lift it, which is the opposite of what most teams assume when they reach for hybrid search.
Fusion is not free improvement
The VLDB 2026 study's "weakest link" finding across 11 real-world datasets means teams should measure each retrieval path's recall on its own before fusing it in — a path with poor recall can drag the combined result below what the best single method achieved alone.
Cost is the other honest constraint. Running lexical, dense and image retrieval in parallel, then fusing, then reranking, multiplies latency and compute for every query — even the simple ones that a single method would have answered correctly. And raw document images are expensive to store in the first place: IRPAPERS reports that storing raw document images for a corpus takes roughly 290 times more storage than storing the equivalent text.
Modality gap is a real bias, not a theoretical one
Forcing text and image into one shared embedding space, rather than routing between modality-specific indexes, biases results toward the query's own modality — UniversalRAG's finding — so a naive multimodal setup can retrieve confidently and still miss the right modality entirely.
Numbers to carry
| Metric | Value |
|---|---|
| Datasets in the VLDB hybrid search study | 11 real-world datasets |
| RRF constant used in a multimodal scientific RAG pipeline | k=25 |
| Reranked contexts kept after fusion in that pipeline | top 3 contexts |
| IRPAPERS hybrid text search recall | 46% Recall@1, 78% Recall@5, 91% Recall@20 |
| IRPAPERS Cohere Embed v4 page image embedding recall | 58% Recall@1, 87% Recall@5, 97% Recall@20 |
| Storage cost of raw document images vs text (IRPAPERS) | roughly 290× more storage |
| Learned orchestration model vs GPT-5.5, Hit@1 (e-commerce) | 0.81 vs 0.69 |
| Learned orchestration model vs GPT-5.5, Hit@1 (email search) | 0.91 vs 0.86 |
Myths worth killing
| Commonly repeated | What is actually true |
|---|---|
| "More retrieval methods always means better answers" | The VLDB 2026 study found a "weakest link" effect: one weak path in the fusion degraded accuracy across 11 datasets rather than improving it |
| "One shared embedding space for text and images is the ideal setup" | UniversalRAG shows a shared embedding space creates a modality gap, biasing results toward the query's own modality; keeping indexes separate and routing between them avoids this |
| "Fusion and reranking do the same job" | Fusion is a coarse, rank-based merge of lists; reranking examines each candidate against the query directly and is only affordable on a small shortlist after fusion |
| "Retrieving the single best page is always the safest choice" | On IRPAPERS, retrieving k=5 pages beat retrieving one page for complex queries, because relevant evidence was often spread across more than one page |
| "Text embeddings are always good enough for scanned documents" | On IRPAPERS, closed-source page image embeddings from Cohere Embed v4 outperformed Voyage 3 Large text embeddings and every tested open-source model |
Worked example
- Meera-style query arrives: "what was the measured error rate in the third experiment's table". The system routes it to lexical, dense-vector and image indexes simultaneously, because the table is likely a scanned image, not typed text.
- Lexical search matches the term "error rate" across typed captions; dense-vector search matches semantically similar phrasing like "measurement deviation"; image-based retrieval (via a multi-vector model such as ColPali) matches the visual layout of a numeric table.
- Each path returns its own ranked list of candidate pages. These are fused with Reciprocal Rank Fusion rather than compared on raw score, since the three paths score on incompatible scales.
- The fused shortlist — illustrative, say a few dozen candidates per method reduced to one combined list — is passed to a cross-encoder reranker, which narrows it to the top 3 highest-scoring pages.
- The system decides depth: rather than handing the generation model only the single top page, it passes the top 5 pages (k=5), because IRPAPERS found this depth outperforms a single page for complex queries.
- The generation model writes an answer citing the scanned table page directly — evidence that a dense-vector-only system would have walked straight past.
Should you use it?
Use hybrid multimodal retrieval when the corpus genuinely mixes formats — scanned pages, tables, figures, free text — and no single retrieval method is getting close to full recall on its own. Skip it when the corpus is homogeneous text and a single well-tuned dense or lexical retriever is already near its recall ceiling, or when the latency budget cannot absorb running several indexes plus a reranker for every query.
The cheapest first experiment is not to build the whole pipeline: measure each retrieval path's recall on its own, on a held-out set of real queries, before fusing anything. If one path is clearly weak, fix or drop it before adding it to a fused ranking that could drag everything else down with it.
Say it in the room
"Measure each retrieval path's recall on its own before fusing — a weak path added to a good one can make the fused answer worse, not better."
Lines to carry
- "More search paths is not automatically better search."
- "Different scores need a common ruler before they can be added."
- "Fusion is a rough sketch; reranking is the fine pencil."
- "One weak retriever can outvote two good ones."
- "It fetches candidates; it never understands them."
- "Measure retrieval before you judge the answer."
FAQ
Isn't hybrid search just running two searches and picking the better one?
No — the harder part is fusion, not running the searches. Two ranked lists on different scales cannot simply be compared; Reciprocal Rank Fusion or a weighted alpha blend has to normalise them before merging, and that fusion step is where naive implementations lose accuracy rather than gain it, per the VLDB 2026 study's "weakest link" finding.
Does adding image search always help a text-heavy corpus?
Only if the corpus actually contains material a text index cannot see — scanned tables, figures, diagrams. On IRPAPERS, closed-source Cohere Embed v4 page image embeddings outperformed hybrid text search and every tested text embedding, but that gap came specifically from documents where relevant evidence lived in images, not typed text.
What's the difference between fusion and reranking?
Fusion merges already-ranked lists using their rank positions, which is cheap but coarse. Reranking takes the fused shortlist and scores each candidate against the query directly with a cross-encoder or tensor-based method, which is more accurate but too expensive to run on anything but a small shortlist.
How many results should I retrieve for the generation model?
More than one, often. IRPAPERS found retrieving k=5 pages beat retrieving a single best page for complex queries, because relevant evidence was frequently spread across more than one page. But depth has a cost: irrelevant pages sneaking into the top-k dilute the generation model's attention rather than help it.
Can a model learn to decide which retrieval paths to run, instead of always running all of them?
That is an active area of work. A learned orchestration model (Qwen3-4B trained with GRPO and DAPO) that decides how to combine hybrid search via executable program synthesis outperformed GPT-5.5 on Hit@1 for e-commerce (0.81 vs 0.69) and email search (0.91 vs 0.86) benchmarks, suggesting orchestration itself can be learned rather than hand-tuned.
Glossary
- Lexical (sparse) search — retrieval based on matching exact words or terms, such as BM25.
- Dense vector search — retrieval that represents queries and documents as single numeric embeddings capturing meaning, matched by nearest neighbour.
- Sparse vector search — retrieval using high-dimensional, mostly-zero vectors derived from term weighting.
- Tensor / multi-vector search — retrieval representing a document as many vectors instead of one, for fine-grained matching at higher cost.
- Reciprocal Rank Fusion (RRF) — combining ranked lists by summing the reciprocal of each item's rank across lists.
- Cross-encoder reranker — a model that scores a query and a candidate document together directly, more accurate but slower than embedding comparison.
- HNSW (Hierarchical Navigable Small World) — a graph-based index for fast approximate nearest-neighbour search over dense vectors.
- BM25 — a classic statistical ranking function for keyword search.
- Modality gap — the tendency of a shared embedding space to retrieve items of the same modality as the query rather than the most relevant item overall.
- Recall@K — the proportion of relevant items found within the top K retrieved results.
- RAG (Retrieval-Augmented Generation) — grounding a generation model's answer by first retrieving relevant external documents.
- Alpha parameter — a tunable weight balancing lexical and vector scores in hybrid fusion.
Sources
- Balancing the Blend: An Experimental Analysis of Trade-offs in Hybrid Search — VLDB Endowment, 2026
- Multimodal Hybrid Retrieval-Augmented Generation for Scientific Document Understanding using Open-Source SLMs — arXiv, 2026
- RAG on Multimodal Databases: Orchestrating Textual, Vector, and Graph-based Retrieval — SBBD, 2025
- UniversalRAG: Retrieval-Augmented Generation over Corpora of Diverse Modalities and Granularities — ACL Anthology, 2026
- IRPAPERS: A Visual Document Benchmark for Scientific Retrieval and Question Answering — alphaXiv, 2026
- HetaRAG: Hybrid Deep Retrieval-Augmented Generation across Heterogeneous Data Stores — alphaXiv, 2025
- ProRetrieval: Learning to Orchestrate Hybrid Search via Executable Program Synthesis — arXiv, 2026