Query Expansion with Locally-Trained Word Embeddings

By Fernando Diaz, Bhaskar Mitra, Nick Craswell. 2016

287 citations recorded by the source corpus.

Continuous space word embeddings have received a great deal of attention in the natural language processing and machine learning communities for their ability to model term similarity and other relationships. We study the use of term relatedness in the context of query expansion for ad hoc information retrieval. We demonstrate that word embeddings such as word2vec and GloVe, when trained globally, underperform corpus and query specific embeddings for retrieval tasks. These results suggest that other tasks benefiting from global embeddings may also benefit from local embeddings.

Subfields: Artificial Intelligence, Information Retrieval & Search

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Retrieval & Search

Citations in the corpus

End-to-End Neural Ad-hoc Ranking with Kernel Pooling (2017, background): “nd that word embeddings trained locally on pseudo relevance feedback documents are more related to the query’s information needs, and can provide beer query expansion terms [5].”
Relevance-based Word Embedding (2017, methodology): “[11] with the general idea of training word embedding models on the top ranked documents retrieved in response to a given query.”
Learning to Match using Local and Distributed Representations of Text for Web Search (2016, methodology): “In IR, a significant number of these works have focused on word embeddings [6, 8, 10, 11, 27, 28, 34, 41] and modelling short-text similarities [15, 16, 29, 35–37].”
Neural Ranking Models with Weak Supervision (2017, methodology): “Word embedding vectors have been applied to term re-weighting in IR models [32, 47], query expansion [10, 33, 43], query classication [25, 44], etc.”
Embedding-based Query Language Models (2016, background): “[9] proposed locally-trained word embeddings for query expansion.”
Neural information retrieval: at the end of the early years (2017, methodology): “Diaz et al. (2016), propose a query expansion language model based on word embeddings learned from topic-constrained corpora.”
Query Expansion Techniques for Information Retrieval: a Survey (2017, methodology): “Reference [84] presented a QE technique based on locally-trained word embedding (such as word2vec and GloVe) for ad hoc information retrieval.”
BERT-QE: Contextualized Query Expansion for Document Re-ranking (2020, methodology): “Some query expansion methods use word embeddings to find the relevant terms to the query (Diaz et al., 2016; Zamani and Croft, 2016).”
A Modern Perspective on Query Likelihood with Deep Generative Retrieval Models (2021, unknown): “This is done for instance by generalizing term salience with translation models [38, 40], and re-weighting terms [57], or through adapting word embeddings for document retrieval by post-filtering [39], retrofitting [16], or re-training on local documents [9].”
NPRF: A Neural Pseudo Relevance Feedback Framework for Ad-hoc Information Retrieval (2018, methodology): “The word embeddings are pre-trained based on a pool of the top 2,000 documents returned by BM25 for individual queries as suggested by (Diaz et al., 2016).”
EXS: Explainable Search Using Local Model Agnostic Interpretability (2018, background): “Recently, the IR community has also seen successful application of deep learning models in document representation [3, 9], query modeling [4, 19, 20] and design of ranking models [6, 10, 11, 16].”
Estimating Embedding Vectors for Queries (2016, background): “[12] proposed to train word embedding vectors on topically-constrained corpora, instead of large topically-unconstrained corpora.”
Large Language Models for Information Retrieval: A Survey (2023, unknown): “Mainly-used methods include relevance feedback [87–92], word-embedding based methods [93, 94] etc.”
Word Embedding Causes Topic Shifting; Exploit Global Context! (2017, background): “[5] suggest training separate word embedding models on the top retrieved documents per query, while Rekabsaz et al.”
Situational Context for Ranking in Personal Search (2017, methodology): “Note that the embedding layers diﬀer from pre-trained word embedding vectors, which have been previously used in [13, 29, 48], in that our embeddings are directly learned from the click training data.”
On the Effect of Low-Frequency Terms on Neural-IR Models (2019, methodology): “Various studies apply neural methods by introducing pre-trained word embeddings into classical IR models [13, 14], adapting word embeddings to retrieval tasks [4, 6], or proposing”
Semantic Speech Retrieval With a Visually Grounded Model of Untranscribed Speech (2017, methodology): “Classic approaches use co-occurrences of words, or resources such as WordNet (Miller, 1995), to expand the query list, while recent methods use word embeddings (Diaz et al., 2016; Roy et al., 2016).”
Riker: Mining Rich Keyword Representations for Interpretable Product Question Answering (2019, methodology): “Word embeddings in our model are initialized with pre-trained results [33] in the general domain, but further trained to be product domain specific [12] and can be more suitable for query expansion in our settings.”
Selective Weak Supervision for Neural Information Retrieval (2020, methodology): “The PRF weak supervision signals in general can promote neural models to similar effectiveness as PRF-based query expansion methods [8, 10, 47].”
COIL: Revisit Exact Lexical Match in Information Retrieval with Contextualized Inverted List (2021, methodology): “Early attempts compute similarity between pre-trained word embedding such as word2vec (Mikolov et al., 2013) and GLoVe (Pennington et al., 2014) to produce matching score (Ganguly et al., 2015; Diaz et al., 2016).”