MergeRUCB: A Method for Large-Scale Online Ranker Evaluation

By Masrour Zoghi, Shimon Whiteson, Maarten de Rijke. 2015

59 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Networks and Communications, Management Science and Operations Research, Information Retrieval & Search, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

In Management Science and Operations Research

In Information Retrieval & Search

In Evaluation & Benchmarks

Citations in the corpus

Contextual Dueling Bandits (2015, methodology): “…(Yue and Joachims, 2011); Sensitivity Analysis of VAriables for Generic Exploration (SAVAGE) (Urvoy et al., 2013); Relative Confidence Sampling (Zoghi et al., 2014a); Relative Upper Confidence Bound (RUCB) (Zoghi et al., 2014b); Doubler, MultiSBM and Sparring (Ailon et al., 2014) and mergeRUCB…”
MergeDTS (2018, methodology): “we may mistakingly infer the wrong ranker preferences. But with too many comparisons we may degrade the user experience as we continue to show results from sub-optimal rankers. Based on previous work [3, 44, 45], the challenge of choosing and comparing rankers can be formalized as a K-armed dueling ”
Exploiting Correlation to Achieve Faster Learning Rates in Low-Rank Preference Bandits (2022, background): “…on generalization to broader models both in terms of distributional parameters (Yue and Joachims, 2009; Gajane et al., 2015; Ailon et al., 2014; Zoghi et al., 2015) as well as combinatorial subset-wise plays (Mohajer et al., 2017; González et al., 2017; Saha and Gopalan, 2018; Sui et al.,…”
Efficient and Optimal Algorithms for Contextual Dueling Bandits under Realizability (2021, background): “Beyond ﬁxed-benchmark regret, notions involving Borda [Busa-Fekete and Hüllermeier, 2014; Jamieson et al., 2015; Falahatgar et al., 2017] and Copeland scores [Zoghi et al., 2015a; Komiyama et al., 2016; Wu and Liu, 2016] have also been considered.”
Copeland Dueling Bandits (2015, result): “Since our results (see §5) establish bounds on the number of queries to non-Copeland winners, they can also be applied to other notions of regret.”
Preference-based Online Learning with Dueling Bandits: A Survey (2018, background): “…gradient descent (Zhao and King, 2016) Stochastic mirror descent (Kumagai, 2017) Voting bandits (Urvoy et al., 2013) Copeland conﬁdence bound (Zoghi et al., 2015a) Copeland winners relative minimum empirical divergence (Komiyama et al., 2016) Double Thompson sampling (Wu and Liu, 2016)…”
Learning to Rank (2005, unknown): “[56, 57, 58, 59] – Challenges associated with deciding which ranker is the best among a population, the concept of Condorcet winner and Relative Upper Confidence Bound algorithm.”