TREC: Experiment and evaluation in information retrieval

By José Luis Vicedo González, Jaime Gómez. 2007

1,118 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Information Systems, Information Retrieval & Search, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Information Retrieval & Search

In Evaluation & Benchmarks

Citations in the corpus

Findings of the 2014 Workshop on Statistical Machine Translation (2014, background): “…P@10), normalized discounted cumulative gain (J¨arvelin and Kek¨al¨ainen, 2002) at 5 and 10 documents (NDCG@5, NDCG@10), mean average precision (MAP) (Voorhees and Harman, 2005), precision reached after R documents retrieved, where R indicates the number of the relevant documents for each query…”
Pretrained Transformers for Text Ranking: BERT and Beyond (2020, neutral): “For a general overview of different aspects of TREC (at least up until the middle of the first decade of the 2000s), the “TREC book” [Voorhees and Harman, 2005] edited by Voorhees and Harman provides a useful starting point.”
Automatic summarising: The state of the art (2007, background): “The topic-with-questions test used for DUC 2005 and 2006 is closely related to one of the forms of question answering studied in the TREC QA evaluations (Voorhees 2005a, 2005b).”
LETOR: A benchmark collection for research on learning to rank for information retrieval (2010, methodology): “These two corpora were selected because (1) they were publicly available [44] and (2) they had been widely used by previous work on ranking in the literature of IR [5,11,10,38].”
Methods for Evaluating Interactive Information Retrieval Systems with Users (2009, background): “For more information about these measures and descriptions of other IR measures, see Voorhees and Harman [288].”
How does clickthrough data reflect retrieval quality? (2008, methodology): “The Cranfield evaluation methodology commonly applied in TREC (see e.g. [23]) uses relevance judgments provided manually by trained experts.”
Relative confidence sampling for efficient on-line ranker evaluation (2014, methodology): “One approach to ranker evaluation is to use manual expert annotations in a TREC-like setting [34].”
Information Retrieval Evaluation (2011, background): “Full coverage of the research is clearly beyond the scope of this lecture and readers are referred to [184] as a general reference and to the full series of online proceedings at http://trec.”