UMass at TREC 2004: Novelty and HARD

By Nasreen Abdul Jaleel, James Allan, W. Bruce Croft, Fernando Diaz, Leah S. Larkey, Xiaoyan Li, Mark D. Smucker, Courtney Wade. 2004

359 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Information Systems, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Natural Language Processing

Citations in the corpus

Pretrained Transformers for Text Ranking: BERT and Beyond (2020, methodology): “at has been shown to improve retrieval effectiveness on average; this is a robust ﬁnding supported by decades of empirical evidence. Query expansion using the RM3 pseudo-relevance feedback technique [Abdul-Jaleel et al., 2004], on top of an initial ranked list of documents scored by BM25, is a popul”
Relevance-based Word Embedding (2017, methodology): “ terms are classi•ed with a high probability as “relevant”. In the next set of experiments, we consider the methods that use the top retrieved documents for query expansion: the relevance model (RM3) [1, 20] as a state-of-the-art pseudo-relevance feedback model, and the local embedding approach rece”
Generation-Augmented Retrieval for Open-Domain Question Answering (2020, methodology): “In Table 4, we show the top-k retrieval accuracy of BM25, BM25 with query expansion (+RM3) (Abdul-Jaleel et al., 2004), DPR (Karpukhin et al., 2020), G AR , and G AR + (G AR +DPR).”
Anserini (2018, methodology): “Anserini provides a reference implementation of the RM3 variant [1] of relevance models [27] using our reranking API.”
Document Expansion by Query Prediction (2019, methodology): “query expansion, we applied the RM3 query expansion technique (Abdul-Jaleel et al., 2004).”
Simple Applications of BERT for Ad Hoc Document Retrieval (2019, methodology): “Building on best practice, query likelihood (QL) with RM3 relevance feed-back (Abdul-Jaleel et al., 2004) provides the initial ranking to depth 1000.”
Keyphrase Generation for Scientific Document Retrieval (2020, methodology): “Overall, BM25 + RM3 achieves the best retrieval e ﬀ ectiveness, conﬁrming previous ﬁndings on ad-hoc retrieval in limited data scenarios (Lin, 2019).”
Overview of the TREC-2014 Microblog Track (2014, methodology): “The “RM3” condition is our reference implementation of the RM3 variant of relevance models [5, 1].”
Document Retrieval Using Entity-Based Language Models (2016, methodology): “We show that our entity-based language models can be used to create eﬀective expanded query forms by “plugging” them into an existing query expansion method: the relevance model [26, 1].”
The Neural Hype and Comparisons Against Weak Baselines (2019, background): “• RM3 variant of relevance models [1]: the number of feedback documents N and the number of feedback terms M from [1, 50] with a step size of 1, the weight of the original query λ from [0, 1] with a step size of 0.”
Large Language Models for Information Retrieval: A Survey (2023, neutral): “Notable examples of such methods include RM3 [83], LCE [84], KL expansion [85], and relevance modeling [86].”
Critically Examining the "Neural Hype": Weak Baselines and the Additivity of Effectiveness Gains from Neural Ranking Models (2019, methodology): “odels. Specifically, we applied neural ranking models on top of the strong baselines that Lin used to make his arguments, which showed that a well-tuned implementation of query expansion based on RM3 [1] beats the average precision reported in two recent neural IR papers, anonymously referred to as ”
SLEDGE: A Simple Yet Effective Baseline for Coronavirus Scientific Knowledge Search (2020, methodology): “05), RM3 query expansion (Jaleel et al., 2004) over default BM25 parameters (feedback terms and feedback docs ∈ [1, 20] by 1), QL Sequential Dependency Model (SDM (Metzler and Croft, 2005), term, ordered, and un-ordered weights by 0.”
Query Expansion Using Word Embeddings (2016, background): “8 is the relevance model RM3 [1].”
Overview of the TREC 2020 Deep Learning Track (2021, methodology): “…based approach— e.g. , Duet [Mitra et al., 2017, Mitra and Craswell, 2019] or using word embeddings [Joulin et al., 2016]—but does not fall into the “nnlm” category • trad: if the run exclusively uses traditional IR methods like BM25 [Robertson et al., 2009] and RM3 [Abdul-Jaleel et al., 2004].”
TREC Deep Learning Track: Reusable Test Collections in the Large Data Regime (2021, methodology): “, Duet [19, 20] or using word embeddings [14]—but does not fall into the “nnlm” category (3) trad: if the run exclusively uses traditional IR methods like BM25 [25] and RM3 [1].”
Simple Techniques for Cross-Collection Relevance Feedback (2019, methodology): “Here, we treat BM25+RM3 as the baseline and the initial ranking.”
Overview of the TREC 2019 deep learning track (2020, methodology): “The best NCG@100 for the document retrieval task is achieved by a well-tuned combination of BM25 [Robertson et al., 2009] and RM3 [Abdul-Jaleel et al., 2004] on top of document expansion using doc2query [Nogueira et al., 2019]—which improves by 22 .”
Pseudo-Relevance Feedback Based on Matrix Factorization (2016, methodology): “, RM3 [1]) outperform other PRF methods, including the regularized mixture model [35] and the divergence minimization model [43].”
The Simplest Thing That Can Possibly Work: Pseudo-Relevance Feedback Using Text Classification (2019, methodology): “In most setups, including the popular RM3 approach widely used today [1], the first k hits of an initial ranked list are assumed to be relevant.”