A study of smoothing methods for language models applied to information retrieval

By Chengxiang Zhai, John D. Lafferty. 2004

1,403 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Information Systems

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

Citations in the corpus

Relevance-based Word Embedding (2017, methodology): “To estimate the relevance model distributions in the training set, we considered top 10 retrieved documents in a target collection in response to each query using the Galago 3 implementation of the query likelihood retrieval model [29] with Dirichlet prior smoothing ( µ = 1500) [45].”
Anserini (2018, methodology): “• LM refers to query likelihood with respect to Dirichlet-smoothed language models [59], derived from the language modeling framework.”
Deep Learning for Biomedical Information Retrieval: Learning Textual Relevance from Click Logs (2017, result): “We compared the performance of our deep learning model against: BM25; the Unigram Query Likelihood Model (UQLM) with Dirichlet Smoothing (Zhai and Lafferty, 2004); Word Mover’s Distance (WMD) that leverages pre-trained word-vectors; and a couple of neural network models based on the architecture…”
A similarity measure for indefinite rankings (2010, background): “…a document: with lower µ values, relative weighting is emphasized, meaning some terms have signi.cantly higher impact than others, whereas with higher µ values, each term tends to have similar weighting and what matters is simply the presence or absence of the term [Zhai and Lafferty 2004].”
Correlated Label Propagation with Application to Multi-label Learning (2006, background): “This property plays a role analogous to smoothing in information retrieval (e.g., [26]).”
Benchmarking Intersectional Biases in NLP (2022, methodology): “Therefore, we use additive smoothing when calculating DI and adjusted DI (Zhai and Lafferty, 2004).”
Monolingual and Cross-Lingual Information Retrieval Models Based on (Bilingual) Word Embeddings (2015, methodology): “It is a unigram language model speciﬁed by the following formula with standard Dirichlet smoothing [46]: where N d is the length of document d from the target collection DC , and µ is the parameter of Dirichlet smoothing.”
Ranking-based clustering of heterogeneous information networks with star network schema (2009, background): “Smoothing [22] is a well-known technology in information retrieval.”
From Neural Re-Ranking to Neural Ranking: Learning a Sparse Representation for Inverted Indexing (2018, methodology): “QL The query likelihood retrieval model [40] with Dirichlet prior smoothing [60].”
UMass at TREC 2004: Novelty and HARD (2004, methodology): “…rank all documents in the collection according to the Kullback-Leibler divergence with P ( w | Q ) , Here, the document language model, P ( w | D ) , may be estimated using a number of different techniques [33]; smoothing parameters used will be described whenever language model retrieval is used.”
TILDE: Term Independent Likelihood moDEl for Passage Re-ranking (2021, neutral): “For example, traditional bag of words [22] and language modelling [29, 30] approaches assume terms are independent.”
Information Retrieval: A Biomedical and Health Perspective (2020, background): “found that younger referees with more experience refereeing had a better chance at detecting flaws in problematic papers [178].”
IRGAN: A Minimax Game for Unifying Generative and Discriminative Information Retrieval Models (2017, background): “Statistical language models of text retrieval consider a reverse generative process from a document to a query: d → q , typically generating query terms from a document (i.e., the query likelihood function) [32, 48].”
Feature location in source code: a taxonomy and survey (2013, methodology): “A case study was conducted on Eclipse in which cognitive assignment was compared to other IR techniques, such as language modeling [68], dependency language model [69], vector space model [16], and LSI.”
A Structural Smoothing Framework For Robust Graph Comparison (2015, background): “The general idea behind smoothing is to adjust the MLE of the probabilities by pushing the high probabilities downwards and pushing low or zero probabilities upwards in order to produce a more accurate distribution on the events [30].”
To Clean or Not to Clean (2018, methodology): “Given this anomalous behaviour of LM-JM, and given that researchers continue to use LM-JM, the recommendations of Zhai and Lafferty [76, 77] notwithstanding (e.”
Integrating and Evaluating Neural Word Embeddings in Information Retrieval (2015, methodology): “is used to express meaning; in addition, it has proven an e↵ective method for document retrieval [30].”