The TREC-8 Question Answering Track Report

By Ellen M. Voorhees. 1999

Subfields: Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding (2018, background): “Other tasks are so close to being solved that evaluation on them is relatively uninformative, such as MPQA (Wiebe et al., 2005) and TREC question classiﬁcation (Voorhees et al., 1999).”
Simple and Effective Multi-Paragraph Reading Comprehension (2017, background): “Open question answering has been the subject of much research, especially spurred by the TREC question answering track (Voorhees et al., 1999).”
Dense Passage Retrieval for Open-Domain Question Answering (2020, methodology): “This approach has been shown to work well previously (Wang et al., 2019, 2018; Lin et al., 2018).”
Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering (2020, background): “While being a longstanding problem in natural language processing (Voorhees et al., 1999), this task has recently regained interest following the work by Chen et al. (2017).”
Few-shot Learning with Retrieval Augmented Language Models (2022, background): “For open-domain question answering (Voorhees et al., 1999), documents are often retrieved from Wikipedia (Chen et al., 2017).”
Distilling Knowledge from Reader to Retriever for Question Answering (2020, background): “Information retrieval is an important component for many natural language processing tasks, such as question answering (Voorhees et al., 1999) or fact checking (Thorne et al., 2018).”
Progressively Pretrained Dense Corpus Index for Open-Domain Question Answering (2020, background): “The problem of answering questions without setting the limits on speciﬁc domains has been intensively studied since the earlier TREC QA competitions (Voorhees, 1999).”
Denoising Distantly Supervised Open-Domain Question Answering (2018, methodology): “The task aims to answer open-domain questions using external resources such as collections of documents (Voorhees et al., 1999), web-pages (Kwok et al., 2001; Chen and Van Durme, 2017), structured knowledge graphs (Berant et al., 2013a; Bordes et al., 2015) or automatically extracted relational…”
AmbigQA: Answering Ambiguous Open-domain Questions (2020, background): “Open-domain Question Answering requires a system to answer any factoid question based on evidence provided by a large text collection such as Wikipedia (Voorhees et al., 1999; Chen et al., 2017).”
Autoregressive Search Engines: Generating Substrings as Document Identifiers (2022, background): “…with knowledge intensive language tasks (Levy et al., 2017; Dinan et al., 2019; Elsahar et al., 2018; Petroni et al., 2021), such as open-domain question answering (Voorhees et al., 1999; Joshi et al., 2017; Yang et al., 2018; Kwiatkowski et al., 2019) and fact checking (Thorne et al., 2018).”
Learning Dense Representations of Phrases at Scale (2020, background): “Open-domain question answering (QA) aims to provide answers to natural-language questions us-ing a large text corpus (Voorhees et al., 1999; Fer-rucci et al., 2010; Chen and Yih, 2020).”
Adaptive Information Seeking for Open-Domain Question Answering (2021, background): “Open-domain question answering (QA) (Voorhees et al., 1999) is a task of answering questions using a large collection of texts (e.g., Wikipedia).”
Cognitive Graph for Multi-Hop Reading Comprehension at Scale (2019, background): “Approaches to get paragraph-level answers has been thoroughly investigated by the information retrieval community, which can be dated back to the 1990s (Belkin, 1993; Voorhees et al., 1999; Moldovan et al., 2000).”