Rank-biased precision for measurement of retrieval effectiveness

By Alistair Moffat, Justin Zobel. 2008

677 citations recorded by the source corpus.

Subfields: Computer Science Applications, Information Systems, Information Retrieval & Search, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Science Applications

In Information Systems

In Information Retrieval & Search

In Evaluation & Benchmarks

Citations in the corpus

Pretrained Transformers for Text Ranking: BERT and Beyond (2020, background): “not have been evaluated fairly. Another approach is to use a metric that directly quantiﬁes the uncertainty in scores that can be attributed to unjudged documents, such as rank-based precision (RBP) [Moffat and Zobel, 2008].54 Obviously, different proportions of unjudged documents is a strong sign t”
Pairwise Crowd Judgments: Preference, Absolute, and Ratio (2018, methodology): “An evaluation metric such as Average Precision (AP) or Rank-Biased Precision (RBP) [12] is then employed, to compute a score for each run based on a set of relevance judgments .”
Novelty and Diversity in Recommender Systems (2015, methodology): “Two normalization approaches are commonly considered in information retrieval metrics, which define ⁄ respectively as: a) the maximum metric value obtainable by an ideal recommendation ranking, e.g. as in nDCG and -nDCG [9], or b) the expected browsing depth, as in RBP [17] and discussed in [10].”
Towards Meaningful Statements in IR Evaluation: Mapping Evaluation Measures to Interval Scales (2021, methodology): “We consider the following evaluation measures: Precision (P) and Recall (R) [93], AP [15], RBP [65], DCG and nDCG [50] and RR [84].”
TOPSIG: topology preserving document signatures (2011, background): “[18] J. Zobel, A. Mo.at, and L.A.F. Park.”
Assessing efficiency–effectiveness tradeoffs in multi-stage retrieval systems without using relevance judgments (2015, background): “…temporally; P () is a query-free scoring regime based on story recency and story importance as determined by non-query features, such as user profile or past reading behavior; and the retrieval system is of the form P (D, k), where k ≈ 10, for example, is the number of stories to be presented.”
Rank and relevance in novelty and diversity metrics for recommender systems (2011, methodology): “85 based on [17]) changes the sign of comparison in several cases.”
Preference-based Evaluation Metrics for Web Image Search (2020, background): “Our experimental results show that the proposed preference-based metric outperforms existing relevancebased metrics, e.g., NDCG and RBP in terms of correlation to SERPlevel preferences.”
Rank-Biased Precision Reloaded: Reproducibility and Generalization (2015, methodology): “The results presented in [12] with TREC-05 are confirmed for all the tested collections showing that RBP.”
A general approximation framework for direct optimization of information retrieval measures (2010, neutral): “See also ( Moffat and Zobel 2008 ) for other measures.”
Information Retrieval for E-Discovery (2013, background): “Such metrics include discounted cumulative gain (DCG), rank-biased precision (RBP), and (as special cases) precision at cutoff k and collection yield [72, 98].”
Expected reciprocal rank for graded relevance (2009, background): “The rank biased precision (RBP) metric recently proposed by Moﬀat and Zobel [17] incorporates a simple user model, but does not directly address the issue described here.”
Score Aggregation Techniques in Retrieval Experimentation (2009, background): “95 (see Moffat & Zobel (2009) for a definition of this metric), and nDCG (see Järvelin & Kekäläinen (2002)) as the underlying similarity measure.”
Fidelity, Soundness, and Efficiency of Interleaved Comparison Methods (2013, background): “Recently developed evaluation metrics that incorporate insights gained from click models [Chapelle et al. 2009; Moffat and Zobel 2008] provide new possibilities for combining click data and editorial judgments, further bridging the gap between click-based and traditional retrieval evaluation.”
Recommender Systems (2012, background): “For a list of length L , the rank-biased precision metric is defined as [108] RBP where r n is the same as in DCG .”
Improvements that don't add up: ad-hoc retrieval results since 1998 (2009, methodology): “The runs are marked up for relevance using the judgments, and the relevance vectors are scored using a measure such as mean average precision (MAP) or rank-biased precision (RBP) [Harman, 1993, Moffat and Zobel, 2008].”
Test Collection Based Evaluation of Information Retrieval Systems (2010, background): “Moffat and Zobel similarly examined error rates when comparing systems on collections with unjudged documents [179].”