Overview of the TREC 2004 Robust Track.

By Ellen M. Voorhees. 2004

154 citations recorded by the source corpus.

Subfields: Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

Pretrained Transformers for Text Ranking: BERT and Beyond (2020, methodology): “action-based and representation-based neural ranking models to a well-engineered implementation of bag-ofwords querying with well-tuned query expansion on the dataset from the TREC 2004 Robust Track [Voorhees, 2004]. Under this limited data condition, most of the neural ranking methods were unable t”
Document Ranking with a Pretrained Sequence-to-Sequence Model (2020, background): “Robust04 (Voorhees, 2004) is the test collection from the TREC 2004 Robust Track.”
Simple Applications of BERT for Ad Hoc Document Retrieval (2019, methodology): “For this, we take advantage of the test collection from the TREC 2004 Ro-bust Track (Voorhees, 2004), which comprises 250 topics over a newswire corpus.”
Simple Techniques for Cross-Collection Relevance Feedback (2019, methodology): “The run leveraged relevance judgments from the TREC 2004 and 2005 Robust Tracks (henceforth, Robust04 and Robust05, respectively).”
Towards Meaningful Statements in IR Evaluation: Mapping Evaluation Measures to Interval Scales (2021, background): “Voorhees [98, 99], in the TREC Robust tracks, explored how to evaluate and compare systems designed to deal with difficult topics and proposed to use the geometric mean, instead of the arithmetic one, for Average Precision (AP) [15].”
Applying BERT to Document Retrieval with Birch (2019, methodology): “For this, we used the test collection from the TREC 2004 Robust Track (Voorhees, 2004), which comprises 250 topics over a newswire corpus of around 500K documents.”
NPRF: A Neural Pseudo Relevance Feedback Framework for Ad-hoc Information Retrieval (2018, methodology): “We evaluate our proposed NPRF framework on two standard test collections, namely, TREC1-3 (Harman, 1993) and Ro-bust04 (Voorhees, 2004).”
Tie-Breaking Bias: Effect of an Uncontrolled Parameter on Information Retrieval Evaluation (2010, background): “Such a comparison is made in Trec’s robust [12] track for characterizing easy and difficult information needs.”
BERT-QE: Contextualized Query Expansion for Document Re-ranking (2020, methodology): “1) A novel query expansion model is proposed to exploit the strength of contextualized model BERT in identifying relevant information from feedback documents; 2) Evaluation on two standard TREC test collections, namely, Robust04 and GOV2, demonstrates that the proposed BERT-QE-LLL could advance the ”
Rank-Biased Precision Reloaded: Reproducibility and Generalization (2015, background): “In the following we consider four public experimental collections, whose characteristics are reported in Table 3: (i) CLEF 2003,Multilingual-4, Ad-Hoc Track [1]; (ii) TREC 13, 2004, Robust Track [15]; (iii) CLEF 2009, bilingual X2EN,The European Library (TEL) Track [7]; and, (iv) TREC 21, 2012, Web ”
Information Retrieval Evaluation (2011, methodology): “However a new metric GMAP using the geometric mean of the average precision scores (as opposed to the arithmetic mean) was used in a variation of the ad hoc track in 2004 to emphasize success on the “harder” topics [187].”
Cross-Domain Sentence Modeling for Relevance Transfer with BERT (2019, methodology): “We conduct end-to-end document ranking experiments on three TREC newswire collections: the Robust Track from 2004 (Robust04) [62] and the Common Core Tracks from 2017 and 2018 (Core17 [1] and Core18 [2]).”