The Power of Noise: Redefining Retrieval for RAG Systems

By Florin Cuconasu, Giovanni Trappolini, F. Siciliano, Simone Filice, Cesare Campagnano, Y. Maarek, Nicola Tonellotto, Fabrizio Silvestri. Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 2024

439 citations recorded by the source corpus.

Retrieval-Augmented Generation (RAG) has recently emerged as a method to extend beyond the pre-trained knowledge of Large Language Models by augmenting the original prompt with relevant passages or documents retrieved by an Information Retrieval (IR) system. RAG has become increasingly important for Generative AI solutions, especially in enterprise settings or in any domain in which knowledge is constantly refreshed and cannot be memorized in the LLM. We argue here that the retrieval component of RAG systems, be it dense or sparse, deserves increased attention from the research community, and accordingly, we conduct the first comprehensive and systematic examination of the retrieval strategy of RAG systems. We focus, in particular, on the type of passages IR systems within a RAG solution should retrieve. Our analysis considers multiple factors, such as the relevance of the passages included in the prompt context, their position, and their number. One counter-intuitive finding of this work is that the retriever's highest-scoring documents that are not directly relevant to the query (e.g., do not contain the answer) negatively impact the effectiveness of the LLM. Even more surprising, we discovered that adding random documents in the prompt improves the LLM accuracy by up to 35%. These results highlight the need to investigate the appropriate strategies when integrating retrieval with LLMs, thereby laying the groundwork for future research in this area.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/77179e5ff669452b9bea479a4236a6e2009ee422

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Bin Cui

Citations in the corpus

Retrieval-Augmented Generation for AI-Generated Content: A Survey (2024, positive): “Though the common sense is that increasing the accuracy of retrieval will contribute to the effectiveness of RAG, a recent study surprisingly shows that noisy retrieval results may conversely help improve the generation quality [369].”
Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools (2024, neutral): “Failing to capture that dimension of hallucination would require us to conclude that a tool that links only to Brown v. Board of Education on every query (or provides cases for fictional judges as in the instance of Luther A. Wilgarten) has provided “hallucination-free” citations, a plainly…”
Retrieval-Augmented Generation for Large Language Models: A Survey (2023, background): “In RAG, the generator’s input encompasses not only typical contextual information but also relevant text segments obtained through the retriever.”
Model Context Protocol (MCP): Landscape, Security Threats, and Future Research Directions (2025, neutral): “Specific API Specific API Specific API”
Evaluation of Retrieval-Augmented Generation: A Survey (2024, background): “These challenges necessitate the development of comprehensive evaluation metrics that can effectively capture the interplay between retrieval accuracy and generative quality [2,6].”
Multi-Agent Collaboration Mechanisms: A Survey of LLMs (2025, neutral): “Their performance degrades as real-world knowledge changes, prompting a focus on aligning models with up-to-date information without retraining from scratch [19, 28].”
Small Encoders Can Rival Large Decoders in Detecting Groundedness (2025, unknown): “As RAG systems surface top-ranked documents, they can still include irrelevant distractions [Cuconasu et al., 2024, Asai et al., 2024, Wang et al., 2023].”
BERGEN: A Benchmarking Library for Retrieval-Augmented Generation (2024, unknown): “While efforts have focused on mitigating hallucinations [9, 28, 60] and dealing with noisy contexts [13] within the LLM component, the impact of the retrieval component to improve responses remains underexplored [5].”