Content-Based Weak Supervision for Ad-Hoc Re-Ranking

By Sean MacAvaney, Andrew Yates, Kai Hui, Ophir Frieder. 2017

46 citations recorded by the source corpus.

One challenge with neural ranking is the need for a large amount of manually-labeled relevance judgments for training. In contrast with prior work, we examine the use of weak supervision sources for training that yield pseudo query-document pairs that already exhibit relevance (e.g., newswire headline-content pairs and encyclopedic heading-paragraph pairs). We also propose filtering techniques to eliminate training samples that are too far out of domain using two techniques: a heuristic-based approach and novel supervised filter that re-purposes a neural ranker. Using several leading neural ranking architectures and multiple weak supervision datasets, we show that these sources of training pairs are effective on their own (outperforming prior weak supervision techniques), and that filtering can further improve performance.

Subfields: Artificial Intelligence, Information Systems

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

Citations in the corpus

Relevance-guided Supervision for OpenQA with ColBERT (2020, background): “…BM25, and existing inner-loop retrieval strategies restrict training the document encoder or require frequent re-indexing and repeated retrieval, RGS offers a scal-2 (Zhang et al., 2020) and headings text (MacAvaney et al., 2019), treated as queries for which the target content is assumed relevant.”
Context-Aware Document Term Weighting for Ad-Hoc Search (2020, methodology): “One limitation of PRF-based approaches is that they rely on the availability of queries and the quality of the pseudo-relevance labels [25].”
Efficient Document Re-Ranking for Transformers by Precomputing Term Representations (2020, neutral): “supervision dataset for training neural models [28].”
Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillation (2020, background): “[30] show the beneficial use of headline - content pairs as pseudo-relevance judgements for weak supervision.”
PROP: Pre-training with Representative Words Prediction for Ad-hoc Retrieval (2020, unknown): “There might be some confusion between the proposed pre-training model PROP and those weak supervision methods [2, 10, 21] in IR, which also leverage some classical models (e.”
Selective Weak Supervision for Neural Information Retrieval (2020, methodology): “The warm up stage first trains the state and action network using the discriminator setup [26].”
MultiCQA: Zero-Shot Transfer of Self-Supervised Text Matching Models on a Massive Scale (2020, methodology): “, 2019b), and similar methods have been used for ad-hoc retrieval (MacAvaney et al., 2019).”