On the history of evaluation in IR

By Stephen E. Robertson. 2008

128 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Information Systems

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

Citations in the corpus

Counterfactual Estimation and Optimization of Click Metrics for Search Engines (2014, background): “There is a long history of evaluation methodology research in the information retrieval community [26].”
Methods for Evaluating Interactive Information Retrieval Systems with Users (2009, methodology): “For a discussion of common evaluation models used in systemscentered research, readers are referred to Spärck-Jones [246] and Robertson [220].”
The Notion of Relevance in Information Science: Everybody knows what relevance is. But, what is it really? (2016, methodology): “Ø Ruthven et al. (2003) used 15 undergraduate and 15 graduate students to search 6 simulated search topics on an experimental and a control system in 5 experiments in which they assessed retrieved documents as to relevance to examine the searchers’ overall search behavior for possibilities of incorp”
Toward Predicting the Outcome of an A/B Experiment for Search Relevance (2015, background): “A lot of work has been done to evaluate rankers in information retrieval and Web search [25].”
Counterfactual Estimation and Optimization of Click Metrics in Search Engines: A Case Study (2015, background): “There is a long history of evaluation methodology research in the information retrieval community [27].”
Economic impact assessment of NIST's text REtrieval conference (TREC) program. Final report (2010, methodology): “The four indexing systems were (1) the Uniterm system, (2) the Universal Decimal Classification (a hierarchical library classification), (3) an alphabetical subject catalogue (subject headings expressed as phrases), and (4) a faceted classification scheme (allowing the construction of complex catego”
Boosted Dense Retriever (2021, methodology): “We are interested to test whether our method could be beneﬁcial for out-of-domain trans-Method EntityQuestions R@20 R@100 BM25 (Chen et al., 2021) 71.2 79.7 DPR (Chen et al., 2021) 49 fer as well.”
Information Retrieval Evaluation (2011, background): “Readers particularly interested in the early history should also see [134, 147, 167].”
Tie-Breaking Bias: Effect of an Uncontrolled Parameter on Information Retrieval Evaluation (2010, background): “The rel ∈ [1, 127] value is assigned to relevant documents.”