Context-Aware Document Term Weighting for Ad-Hoc Search

By Zhuyun Dai, Jamie Callan. 2020

124 citations recorded by the source corpus.

Bag-of-words document representations play a fundamental role in modern search engines, but their power is limited by the shallow frequency-based term weighting scheme. This paper proposes HDCT, a context-aware document term weighting framework for document indexing and retrieval. It first estimates the semantic importance of a term in the context of each passage. These fine-grained term weights are then aggregated into a document-level bag-of-words representation, which can be stored into a standard inverted index for efficient retrieval. This paper also proposes two approaches that enable training HDCT without relevance labels. Experiments show that an index using HDCT weights significantly improved the retrieval accuracy compared to typical term-frequency and state-of-the-art embedding-based indexes.

Subfields: Artificial Intelligence, Information Systems, Information Retrieval & Search

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Information Retrieval & Search

Citations in the corpus

Autoregressive Search Engines: Generating Substrings as Document Identifiers (2022, background): “…on term-weighting schemes, with many recent works proposing sophisticated, contextualized weights for both queries, and documents(Dai and Callan, 2019; Gao et al., 2021; Lin and Ma, 2021; Mallia et al., 2021; Dai and Callan, 2020; Bai et al., 2020; Zhao et al., 2021; Formal et al., 2021b,a).”
Pretrained Transformers for Text Ranking: BERT and Beyond (2020, background): “On top of multi-stage ranking architectures, researchers have proposed additional innovations, including document expansion [32, 34] and term importance prediction [3, 5].”
Sparse, Dense, and Attentional Representations for Text Retrieval (2020, methodology): “, 2019b) and DOCT5QUERY (Nogueira and Lin, 2019), which use neural models to expand documents before indexing and scoring with sparse retrieval models, 3) DEEPCT (Dai and Callan, 2020b), which learns to map BERT’s contextualized text representations to context-aware term weights, 4) HDCT (Dai and Ca”
Jointly Optimizing Query Encoder and Product Quantization to Improve Retrieval Performance (2021, background): “• HDCT [14]: It extends DeepCT [13] to support long documents.”
Complementing Lexical Retrieval with Semantic Residual Embedding (2020, methodology): “We set k1 and b in BM25 and DeepCT using values recommended by Dai and Callan (2020b), which show stronger performance”
SPARTA: Efficient Open-Domain Question Answering via Sparse Transformer Matching Retrieval (2020, unknown): “, 2020) and DeepCT (Dai and Callan, 2020).”
SparTerm: Learning Term-based Sparse Representation for Fast Text Retrieval (2020, methodology): “s to adapt the max length (256) of the sequence of our model and generate the sparse representation of each passage with our model. We compare our models with two baseline methods: BM25 [14] and HDCT [3]. HDCT is based on the work of DeepCT and focuses on document ranking, which is also a term weigh”
Learning Passage Impacts for Inverted Indexes (2021, background): “DeepCT [4], for passages, andHDCT [5], for documents, estimate a term’s context-specific importance in each passage/document, by projecting each word’s BERT representation into a single term weight.”
Dense Text Retrieval Based on Pretrained Language Models: A Survey (2022, methodology): “For example, sparse retrieval models can be utilized to provide initial search results for training dense retrieval models [1], [204], while PLMs can be also used to learn term weights to improve sparse retrieval [167], [168] (as discussed in Section 4.”
COIL: Revisit Exact Lexical Match in Information Retrieval with Contextualized Inverted List (2021, methodology): “For DeepCT and DocT5Query, we use the rankings provided by the authors.”
Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillation (2020, methodology): “In order to reduce the query processing latency, Mackenzie et al. [31] propose a static index pruning method when augmenting the inverted index with precomputed re-weighted terms [8].”
Efficiency Implications of Term Weighting for Passage Retrieval (2020, methodology): “DeepCT-Index was shown to significantly outperform classic tf retrieval methods in the context of both passage ranking [5] and web search [6].”