A similarity measure for indefinite rankings

By William Webber, Alistair Moffat, Justin Zobel. TOIS, 2010

884 citations recorded by the source corpus.

Subfields: Information Systems, Management Science and Operations Research

View Paper: https://www.semanticscholar.org/paper/b82756edb00e92f7314f1a3e036bf292664ad3a5

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Systems

In Management Science and Operations Research

Citations in the corpus

Assessing Top- Preferences (2020, methodology): “In creating RBO,Webber et al. [33] carefully identi ed and speci ed the requirements of rank similarity for what they call inde nite rankings, such as the output of rankers.”
Evaluation Measures Based on Preference Graphs (2021, background): “[28] explicitly created RBO to match the requirements of search and related applications.”
Benchmarking Cognitive Biases in Large Language Models as Evaluators (2023, methodology): “We calculated the Rank-Biased Overlap (RBO) score (Webber et al., 2010) to measure the agreement between human preferences and model evaluations in ranking-generated texts across 16 different LLMs. RBO, which can vary from 0 (non-conjoint) to 1 (identical), assigns more weight to the top k items in…”
What drives study‐dependent differences in distance–decay relationships of microbial communities? (2021, background): “…perform well when appropriate (centred log- ratio) transformations are applied to counts (Gloor et al., 2017), or recently developed metrics, such as the rank bias overlap index, show promise for analysing similarity between communities based on species abundance ranks (Webber et al., 2010).”
“I updated the ”: The evolution of references in the English Wikipedia and the implications for altmetrics (2020, methodology): “We also consider the positions of each editor in the two main rankings (most active Wiki-pedians and the 10,000 most active editors of references) by looking at the rank-biased overlap (RBO) by Webber, Moffat, and Zobel (2010).”
Pre-training is a Hot Topic: Contextualized Document Embeddings Improve Topic Coherence (2020, methodology): “We deﬁne ρ as the reciprocal of the standard RBO (Webber et al., 2010; Terragni et al., 2021b).”
Assessing efficiency–effectiveness tradeoffs in multi-stage retrieval systems without using relevance judgments (2015, methodology): “For example, a BM25 or Language Model similarity computation based on collection, term, and document statistics might be used to score each document in the collection [see Zobel and Moffat (2006) or Büttcher et al.”
Ranking consistency for image matching and object retrieval (2014, background): “The rank biased overlap (RBO) similarity [32] for a truncated ranking list is defined as”
Query Driven Algorithm Selection in Early Stage Retrieval (2017, methodology): “In this work, we explore how to apply a reference list framework [13, 45, 46, 55] to alleviate this problem.”
Tranco: A Research-Oriented Top Sites Ranking Hardened Against Manipulation (2018, background): “Finally, Quantcast’s removal of non-quantiﬁed sites after November 14, 2018 causes a signiﬁcant drop in RBO to less than 5.5%, with no overlap of the top 10: many very popular domains are not quantiﬁed and are therefore now missing from Quantcast’s list.”
flexvec: SQL Vector Retrieval with Programmatic Embedding Modulation (2026, unknown): “Three metrics used to measure behavioral change: • RBO (Rank-Biased Overlap) [12]: similarity between two ranked lists, weighted toward the top.”