Reciprocal rank fusion outperforms condorcet and individual rank learning methods

By Gordon V. Cormack, C. Clarke, Stefan Büttcher. 2009

1,081 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Information Systems, Information Retrieval & Search

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Information Systems

In Information Retrieval & Search

Citations in the corpus

Pretrained Transformers for Text Ranking: BERT and Beyond (2020, methodology): “sources of fusion involved variants of BERT-based models, or transformer-based rerankers applied to different ﬁrst-stage retrieval approaches. Simple fusion techniques such as reciprocal rank fusion [Cormack et al., 2009] or linear combinations [Vogt and Cottrell, 1999] were effective and robust, wi”
Exploring and Predicting Transferability across NLP Tasks (2020, methodology): “Then, the aggregated score is computed according to the reciprocal rank fusion algorithm (Cormack et al., 2009): RRF(s) = c∑ i=1 1 60 + ri .”
Multi-Stage Document Ranking with BERT (2019, methodology): “The B I - NARY method is inspired by the Condorcet method (Montague and Aslam, 2002), which is a strong aggregation baseline (Cormack et al., 2009).”
Cross-Domain Sentence Modeling for Relevance Transfer with BERT (2019, methodology): “3686 AP [15] with a non-neural method based on ensembles, which has stood unchallenged for ten years.”
Review-based Question Generation with Adaptive Instance Transfer and Augmentation (2019, methodology): “QG+QR and QA-LSTM via merging the ranking lists of the two models with Reciprocal Rank Fusion (Cormack et al., 2009).”
Overview of Touché 2022: Argument Retrieval (2022, positive): “For the final ranking, the team experimented with re-ranking (up to top-1000 documents from the initial ranking) based on the argument quality by multiplying the relevance and the quality scores and Reciprocal Ranking Fusion [20].”
Critically Examining the "Neural Hype": Weak Baselines and the Additivity of Effectiveness Gains from Neural Ranking Models (2019, background): “The highest AP we encountered was by Cormack et al. [4] in 2009, at 0.3686.”
Covidex: Neural Ranking Models and Keyword Search Infrastructure for the COVID-19 Open Research Dataset (2020, methodology): “Our covidex.t5 run takes r2.fusion1 and r2.fusion2 , reranks both with monoT5-3B, and then combines (with RRF) the outputs of both.”
What to Pre-Train on? Efficient Intermediate Task Selection (2021, methodology): “(Cormack et al., 2009) to aggregate the rankings of TextEmb and TaskEmb. further experiment with various combinations of ranks produced by methods of different categories, e.g. Size + SEmb .”
Benchmarking Retrieval-Augmented Generation for Medicine (2024, methodology): “Additionally, we utilize Reciprocal Rank Fusion (RRF, Cormack et al., 2009) to combine results from different retrievers, including RRF-2 (fusion of BM25 and MedCPT), and RRF-4 (fusion of all four retrievers).”
SemEval-2016 Task 3: Community Question Answering (2019, methodology): “We used several different query/document formulations to perform 21 retrieval runs, and we merged the retrieved results, ranking them according to the reciprocal rank fusion algorithm (Cormack et al., 2009).”
Cross-Lingual Training of Neural Models for Document Ranking (2020, methodology): “Alternatively, we also experimented with Reciprocal Rank Fusion (Cormack et al., 2009) to aggregate two separate ranked lists, which is denoted ENS RRF .”
Preferences on a Budget: Prioritizing Document Pairs when Crowdsourcing Relevance Judgments (2022, methodology): “• Reciprocal Rank Fusion (RRF) : RRF is a method for combining the rankings from multiple systems which demonstrated to outperform individual rank learning methods [11, 17].”
Cross-Domain Modeling of Sentence-Level Evidence for Document Retrieval (2019, methodology): “5 Furthermore, our results exceed the previous highest known score of 0.3686, which is a non-neural method based on ensembles (Cormack et al., 2009).”
Revisiting User Mobility and Social Relationships in LBSNs: A Hypergraph Embedding Approach (2019, methodology): “Similar to the friendship prediction task, we combine the results from mobility and social features using reciprocal rank fusion [9], resulting in four combinations (i.”
What Makes a Good Fiqh Retriever? Answer Retrieval for Arabic Islamic Jurisprudence (2026, unknown): “For hybrid retrieval, the top 50 candidates from both the dense retriever and BM25 are fused using Reciprocal Rank Fusion (RRF) with a rank constant of 60.”
Towards Retrieval Augmented Generation in High-Energy and Astroparticle Physics (2026, unknown): “We use k = 60 which is a standard RRF configuration [22].”