TREC Complex Answer Retrieval Overview

By Laura Dietz, Manisha Verma, Filip Radlinski, Nick Craswell. 2018

171 citations recorded by the source corpus.

Subfields: Information Retrieval & Search

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Retrieval & Search

Citations in the corpus

Passage Re-ranking with BERT (2019, background): “Introduced by Dietz et al. (2017), in this dataset, the input query is the concatenation of a Wikipedia article title with the title of one of its section.”
ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT (2020, methodology): “as baselines in Section 4.3. We do so to avoid submi−ing multiple variants of the same model at once, as the organizers discourage too many submissions by the same team. TREC CAR. Introduced by Dietz [7] et al. in 2017, TREC CAR is a synthetic dataset based on Wikipedia. It consists of about 29M pas”
Sparse, Dense, and Attentional Representations for Text Retrieval (2020, background): “, 2019a), especially since largescale annotated data has become available for training deep neural models (Dietz et al., 2018; Craswell et al., 2020).”
Relevance-guided Supervision for OpenQA with ColBERT (2020, background): “, 2016) and the TREC CAR (Dietz et al., 2017) search collections.”
Context-Aware Sentence/Passage Term Importance Estimation For First Stage Retrieval (2019, methodology): “TREC-CAR [12] consists of 29 .”
Multi-Stage Document Ranking with BERT (2019, methodology): “For ﬁne-tuning, we generate our query– document pairs by retrieving the top ten documents from the entire TREC CAR corpus using BM25.”
Document Expansion by Query Prediction (2019, positive): “The test set is the same used to evaluate the submissions to TREC-CAR 2017 (approx.”
Expansion via Prediction of Importance with Contextualization (2020, methodology): “tency and those that trade off latency of effectiveness (e.g., BERT Large). To test whether EPIC is effective on other passage ranking tasks as well, we test on the TREC CAR passage ranking benchmark [4]. When trained and evaluated on the 2017 dataset (automatic relevance judgments) with BM25, the M”
Cross-Domain Modeling of Sentence-Level Evidence for Document Retrieval (2019, unknown): “…of test collections that fortuitously contain passage-level relevance evidence: the MA-chine Reading COmprehension (MS MARCO) dataset (Bajaj et al., 2018), the TREC Complex Answer Retrieval (CAR) dataset (Dietz et al., 2017), and the TREC Microblog (MB) datasets (Lin et al., 2014), described below.”
A Wrong Answer or a Wrong Question? An Intricate Relationship between Question Reformulation and Answer Selection in Conversational Question Answering (2020, methodology): “After de-duplication, the MS MARCO collection contains 8.6M documents and the TREC CAR – 29.8M documents.”
Interpretable & Time-Budget-Constrained Contextualization for Re-Ranking (2020, methodology): “We use a vocabulary of all terms with a minimum collection occurrence of 5 for MSMARCO-Passage and TREC CAR; and a collection minimum of 10 for MSMARCO-Document as it contains more unique terms.”
Multi-agent query reformulation: Challenges and the role of diversity (2019, methodology): “Following Dietz et al. (2017), we report the results on four standard TREC evaluation measures: RPrecision (R-Prec), Mean-average Precision (MAP), Reciprocal Rank (MRR), and Normalize Discounted Cumulative Gain (NDCG).”
A Deep Look into Neural Ranking Models for Information Retrieval (2019, background): “Standard benchmark datasets [36, 37], evaluation tasks [38], and open-source toolkits [39] have been created to facilitate research and rigorous comparison.”
Applying BERT to Document Retrieval with Birch (2019, methodology): “Furthermore, it appears that out-of-domain passage-level relevance judgments fortu-itously available, such as the MS MARCO passage dataset (Bajaj et al., 2018) and the TREC CAR dataset (Dietz et al., 2017), are also beneﬁcial.”
CAsT-19: A Dataset for Conversational Information Seeking (2020, methodology): “The corpus is passages from MS MARCO1 and the TREC Complex Answer Retrieval Paragraph Collection [4].”
Training Curricula for Open Domain Answer Re-Ranking (2020, methodology): “1), TREC Complex Answer Retrieval (CAR) [10] (Section 4.”
MS MARCO: Benchmarking Ranking Models in the Large-Data Regime (2021, methodology): “Yilmaz et al. [85] finetune a BERT-based [21] model on MS MARCO, TREC CAR [22] and TREC Microblog [43] datasets and evaluate them on three TREC newswire collections: Robust04 [74], Core17 [1], and Core18.”
Overview of the TREC 2019 deep learning track (2020, background): “Dietz et al. [2017] have a TREC task with enough training data to investigate such ﬁndings, but on synthetic rather than human-labeled data.”
Cross-Domain Sentence Modeling for Relevance Transfer with BERT (2019, methodology): “We fine-tune BERT on three such sentenceand passage-level datasets individually and in combination: TREC Microblog [47], MicroSoft MAchine Reading Comprehension [6] and TREC Complex Answer Retrieval [20].”