TREC 2017 Common Core Track Overview

By James Allan, Donna K. Harman, Evangelos Kanoulas, Dan Li, Christophe Van Gysel, Ellen M. Voorhees. 2017

79 citations recorded by the source corpus.

Subfields: Information Retrieval & Search, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Retrieval & Search

In Evaluation & Benchmarks

Citations in the corpus

Pretrained Transformers for Text Ranking: BERT and Beyond (2020, background): “• Topics and relevant judgments from the TREC 2018 Common Core Track [Allan et al., 2018], which used a corpus of 600K articles from the TREC Washington Post Corpus.”
Document Ranking with a Pretrained Sequence-to-Sequence Model (2020, background): “Core17 (Allan et al., 2017) is the test collection from the TREC 2017 Common Core Track, with relevance judgments for 50 topics on ∼1.”
Effective User Interaction for High-Recall Retrieval: Less is More (2018, methodology): “We used the TREC 2017 Common Core Track [3] test collection for our search topics and documents.”
Towards Meaningful Statements in IR Evaluation: Mapping Evaluation Measures to Interval Scales (2021, background): “• Common Core track T26 [3]: 1,855,658 documents of the New York Times corpus; T26 provides 50 topics, each with multi-graded relevance judgments (not relevant, relevant, highly relevant); relevance judgements were done mixing depth-10 pools with multi-armed bandit approaches [57, 100]; 75 system ru”
A System for Efficient High-Recall Retrieval (2018, methodology): “We used the 50 NIST topics and the corpus provided by the TREC 2017 Common Core Track [1].”
Simple Techniques for Cross-Collection Relevance Feedback (2019, methodology): “Another more recent example is the TREC 2017 Common Core Track [2], a renewed effort to focus on the classic ad hoc retrieval task, which also reused topics from the TREC 2004 Robust Track.”
Cross-Domain Sentence Modeling for Relevance Transfer with BERT (2019, methodology): “Chapter 5 presents our results on three newswire collections—Robust04, Core17 and Core18—and provides further analysis to help interpret the inner workings of BERT.”
The Effect of Content-Equivalent Near-Duplicates on the Evaluation of Search Engines (2020, methodology): “1 Thus validated, we go on to apply it to a selection of ad hoc retrieval tracks succeeding the one originally studied by Bernstein and Zobel: the Terabyte Track (2004-2006) [7,12,6], the Web Track (20092014) [8,9,10,11,13,14], and the ongoing Common Core Track (2017-2018) [1,2].”
Cross-Domain Modeling of Sentence-Level Evidence for Document Retrieval (2019, unknown): “Our best model also achieves a higher AP on Core17 than the best TREC submission that does not make use of past labels or human intervention ( umass baselnrm , 0.275 AP) (Allan et al., 2017).”