Overview of the TREC 2003 Question Answering Track

By Ellen M. Voorhees. 2004

742 citations recorded by the source corpus.

Subfields: Information Retrieval & Search, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Retrieval & Search

In Evaluation & Benchmarks

Citations in the corpus

An efficient framework for learning sentence representations (2018, neutral): “…sentiment (MR) (Pang & Lee, 2005), product reviews (CR) (Hu & Liu, 2004), subjectivity classiﬁcation (SUBJ) (Pang & Lee, 2004), opinion polarity (MPQA) (Wiebe et al., 2005), question type classiﬁ-cation (TREC) (Voorhees & Buckland, 2003) and paraphrase identiﬁcation (MSRP) (Dolan et al., 2004).”
ELI5: Long Form Question Answering (2019, methodology): “Extractive QA Extractive question answering datasets such as TREC (Voorhees, 2003), SQuAD (Rajpurkar et al., 2016, 2018), NewsQA (Trischler et al., 2017), SearchQA (Dunn et al., 2017), and QuAC (Choi et al., 2018) constrain the answer to a word or short phrase from the input and evaluate using exact”
Towards Coherent and Engaging Spoken Dialog Response Generation Using Automatic Conversation Evaluators (2019, positive): “The models are used to get sentence embeddings, which are then passed through feedfor-ward networks that are trained for the following classiﬁcation tasks: (i) Semantic Textual Similarity (STS) (Marelli et al., 2014), (ii) Question Type Classiﬁcation (TREC) (Voorhees and Dang, 2003), (iii) Subjectiv”
Answering Clinical Questions with Knowledge-Based and Statistical Techniques (2007, methodology): “Nugget F-score has been employed as a metric in the TREC question answering track since 2003, to evaluate so-called definition and “other” questions (Voorhees, 2003).”
Question Classification by Weighted Combination of Lexical, Syntactic and Semantic Features (2011, background): “Some early studies build question classifiers based on matching with hand-crafted rules [15].”
Learning strategies for story comprehension: a reinforcement learning approach (2005, background): “Most TREC-style QA systems are designed to extract an answer from a document contained in a fairly large general collection [Voorhees, 2003].”
Evaluating and predicting answer quality in community QA (2010, background): “One significant distinction, however, is that in studying CQA there is an actual user, while the questions used in the TREC QA track, for example, are questions submitted to the FAQ Finder system, and the found answers (by participating sites) are evaluated by trained assessors [15].”
When will information retrieval be "good enough"? (2005, result): “It is similar in spirit to list and definition questions in the TREC question answering task [20].”
Test Collection Based Evaluation of Information Retrieval Systems (2010, background): “Following on from this, the novelty track of TREC [103] and the QA track [272] both encouraged the building of systems that retrieved fragments of documents (sentences for the novelty track, so-called nuggets for QA) that were both relevant and had not previously been seen.”
UvA-DARE ( Digital Academic Repository ) Using Wikipedia at the TREC QA Track (2005, background): “Many systems participating in the TREC QA track use not only the local (AQUAINT) corpus, but also additional knowledge sources such as the web and various gazetteers [15].”
Natural Language Questions for the Web of Data (2012, methodology): “State-of-the-art methods (Hirschman and Gaizauskas, 2001; Kwok et al., 2001; Zheng, 2002; Katz et al., 2007; Dang et al., 2007; Voorhees, 2003) cast the user’s question into a keyword query to a Web search engine (perhaps with phrases for location and person names or other proper nouns).”
Evita: A Robust Event Recognizer For QA Systems (2005, methodology): “Then, subsections 2.2 to 2.5 provide a detailed description of Evita’s different subcomponents: preprocessing, clustering of chunks, event identiﬁcation, and analysis of the grammatical features associated to events.”