Fine-tuning is not the answer to a bad retrieval system.

The failure mode arrives quietly. A RAG system goes live, early demos look good — clean, confident answers with cited sources. Two months in, support tickets start: the system is getting things wrong, not obviously, just missing a clause, conflating two policies, citing the procedure for the wrong machine model. The team looks at the model — tries a larger one, a fine-tuned variant, a prompt template with more instruction about accuracy. Some of it moves the numbers slightly. None of it fixes the problem sustainably, because the problem was never in the model. It was in the retrieval layer — specifically, in what got passed to the model in the first place. A model can only generate from what it receives, and fine-tuning a model on top of broken retrieval doesn't fix the retrieval. It teaches the model to be more articulate about the wrong information.

The most dangerous property of retrieval failure is that it looks like success. A dense retrieval system returning the wrong chunk doesn't error out — it returns a similarity score, typically 0.7 to 0.9, that the generation layer treats as a confidence signal. A score of 0.84 means "this is probably relevant." It does not mean "this is the correct chunk for this query." The generation layer doesn't know the difference.

The research is unambiguous on the scale of this. Dense retrievers underperform BM25 on out-of-domain corpora by 11.7 NDCG points on average across the BEIR benchmark. Single-vector retrieval surfaces all the evidence needed to answer a multi-hop question only 44% of the time on HotpotQA. These aren't edge cases — they're the normal operating conditions of most enterprise RAG deployments, running on heterogeneous corpora with specialised terminology the embedding model was never trained on, where a meaningful share of real queries need evidence from more than one document.

Retrieval failures cluster around a small set of structural causes, each with a diagnostic signature visible if you look at the right layer:

Chunking mismatched to document structure. Uniform token splitting is the default in most implementations and wrong for most enterprise document types. A policy document split on 512-token boundaries routinely cuts a numbered clause in half — the condition lands in one chunk, the consequence in the next, and neither retrieves well for a query about the rule. This isn't a model problem. It's a chunking problem, fixed with content-aware splitting: section-aware for policy documents, step-preserving for procedures.

Dense-only retrieval on entity-heavy queries. Dense retrievers capture semantic similarity, not exact term overlap. A query about a specific contract clause number or named regulation gets matched to something semantically close — and close is wrong. BM25, operating on term frequency, handles exact entity queries precisely. A system routing everything through a single dense encoder is systematically failing on a class of queries that makes up a real share of enterprise traffic. The fix is hybrid retrieval — dense for semantic queries, BM25 for entity queries, weighted fusion between them.

Single-vector retrieval on multi-hop questions. Some questions need evidence from more than one document — approval thresholds that depend on a procurement policy, a data governance framework, and a legal addendum that overrides both. A single query vector finds the most similar chunk, probably the procurement policy, and the model generates an answer that sounds complete and omits the rest. The fix is query decomposition: breaking the multi-part question into sub-queries that each retrieve their own evidence before synthesis.

None of these are fixed by a bigger model or a fine-tuning pass. They're fixed by looking at the retrieval layer first — because that's where the diagnosis almost always belongs, and it's the step most teams skip on their way to the part that feels like it requires more sophisticated engineering.

Further reading: Thakur, N. et al. (2021). BEIR. NeurIPS. arXiv:2104.08663. · Yang, Z. et al. (2018). HotpotQA. EMNLP. arXiv:1809.09600. · Cormack, G.V., Clarke, C.L.A., & Buettcher, S. (2009). Reciprocal Rank Fusion. SIGIR. · Gao, L. et al. (2022). Precise Zero-Shot Dense Retrieval (HyDE). arXiv:2212.10496.