Improvements to BM25 and Language Models Examined

By Andrew Trotman, Antti Puurula, Blake Burgess. 2014

278 citations recorded by the source corpus.

Subfields: Information Retrieval & Search

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Retrieval & Search

Citations in the corpus

Pretrained Transformers for Text Ranking: BERT and Beyond (2020, neutral): “0.186 in terms of MRR@10, but the Microsoft BM25 baseline scores two points lower at 0.165. Non-trivial differences in “BM25 rankings” have been observed by different researchers in multiple studies [Trotman et al., 2014, Mühleisen et al., 2014, Kamphuis et al., 2020]. There are a number of reasons ”
Anserini (2018, background): “To complicate matters, Trotman et al. [52] noted that there are at least half a dozen different variants of scoring functions commonly referred to as BM25 or query likelihood.”
Anserini: Enabling the Use of Lucene for Information Retrieval Research (2017, background): “Trotman et al. [15] pointed out that BM25 and query likelihood with Dirichlet priors can actually refer to at least half a dozen variants, and in some cases, differences in effectiveness are statistically significant.”
Toward Reproducible Baselines: The Open-Source IR Reproducibility Challenge (2016, background): “Trotman et al. [17] pointed out that BM25 and query likelihood with Dirichlet smoothing can actually refer to at least half a dozen diﬀerent variants; in some cases, diﬀerences in eﬀectiveness are statistically signiﬁcant.”
Which BM25 Do You Mean? A Large-Scale Reproducibility Study of Scoring Variants (2020, methodology): “001, the same as Trotman et al. [11].”
Supporting Interoperability Between Open-Source Search Engines with the Common Index File Format (2020, result): “In short, we have replicated previous replicability studies [6, 14], but in a manner that supports cross-system comparisons.”
The Neural Hype and Comparisons Against Weak Baselines (2019, background): “[14] cataloged at least half a dozen variants of what researchers have referred to generically as “BM25” or “query likelihood with Dirichlet smoothing”.”
Building Legal Reward Models for Grounding and Abstention (2026, unknown): “On unanswerable contexts, refusal is largely determined by whether the hard negatives are topically plausible: off-topic random or GloVe negatives are well-rejected, while BM25 and dense semantic negatives are confus-able.”
Retrieval And Structuring Augmented Generation with Large Language Models (2025, unknown): “Several statistical variations of BM25 have been introduced [240].”
CLARC: C/C++ Benchmark for Robust Code Search (2026, unknown): “BM25 (Trotman et al., 2014) BM25 calculates a relevance score for each function by considering the frequency of query terms within that function (Term Frequency), the inverse frequency of those query terms across the entire code collection (Inverse Document Frequency or IDF), and the function’s leng”
CoRet: Improved Retriever for Code Editing (2025, unknown): “A solution is to use negatives from lexical models such as BM25 during training (Karpukhin et al., 2020; Gao et al., 2021; Luan et al., 2021).”