Relative confidence sampling for efficient on-line ranker evaluation

By Masrour Zoghi, Shimon Whiteson, Maarten de Rijke, R´emi Munos. 2014

67 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Networks and Communications, Management Science and Operations Research, Information Retrieval & Search

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

In Management Science and Operations Research

In Information Retrieval & Search

Citations in the corpus

Dueling RL: Reinforcement Learning with Trajectory Preferences (2021, methodology): “Over the last two decades the problem of learning from preference feedback in bandits, known as dueling bandits, has gained much attention (Yue et al., 2012; Zoghi et al., 2014b, 2015).”
Contextual Dueling Bandits (2015, methodology): “…(Yue and Joachims, 2011); Sensitivity Analysis of VAriables for Generic Exploration (SAVAGE) (Urvoy et al., 2013); Relative Confidence Sampling (Zoghi et al., 2014a); Relative Upper Confidence Bound (RUCB) (Zoghi et al., 2014b); Doubler, MultiSBM and Sparring (Ailon et al., 2014) and mergeRUCB…”
MergeDTS (2018, methodology): “This is a standard approach to evaluating dueling bandit algorithms [47, 50, 56].”
Preference-based Online Learning with Dueling Bandits: A Survey (2018, methodology): “The RUCB algorithm achieves a high probability regret bound of order K log T by merely assuming the existence of a Condorcet winner.”
Towards Conversational Recommender Systems (2016, methodology): “The method picks A exactly as in and for item B, inspired by the dueling bandit approach in [34], it picks the item with the largest probability of being preferred to item A.”
Double Thompson Sampling for Dueling Bandits (2016, methodology): “Consider the following extreme case in Condorcet dueling bandits: assume arm-1 is the Condorcet winner with p1j = 0.501 for all j > 1, and arm-2 is not the Condorcet winner, but with p2j = 1 for all j > 2 (in fact, arm-2 is the Borda winner for larger K [20]).”
Active Evaluation: Efficient NLG Evaluation with Few Pairwise Comparisons (2022, methodology): “For1293 the RCS algorithm, described in algorithm 1 of1294 (Zoghi et al., 2014a), we use α (exploratory con-1295 stant) = 0.501.”
Principled Reinforcement Learning with Human Feedback from Pairwise or K-wise Comparisons (2023, background): “In the literature of dueling bandit , one compares two actions and aims to minimize regret based on pairwise comparisons (Yue et al., 2012; Zoghi et al., 2014b; Yue and Joachims, 2009, 2011; Saha and Krishnamurthy, 2022; Ghoshal and Saha, 2022; Saha and Gopalan, 2018a; Ailon et al., 2014; Zoghi et…”
Relative Upper Confidence Bound for the K-Armed Dueling Bandit Problem (2013, unknown): “Since the submission of this paper, the latter of these two ideas has been validated experimentally in (Zoghi et al., 2014), although a theoretical analysis is still lacking.”
Regret Lower Bound and Optimal Algorithm in Dueling Bandit Problem (2015, methodology): “We compared the IF, BTM withγ = 1.2, RUCB withα = 0.51, Condorcet SAVAGE withδ = 1/T , MultiSBM and Sparring withα = 3, and RMED algorithms.”
Copeland Dueling Bandits (2015, unknown): “In this paper, we consider a formulation of the problem that does not assume the existence of a Condorcet winner.”
MergeRUCB: A Method for Large-Scale Online Ranker Evaluation (2015, methodology): “Relative upper conﬁdence bound (RUCB) [34] and relative con-ﬁdence sampling (RCS) [33] are two related algorithms that select rankers to compare by ﬁrst choosing a ranker that is believed to be a good candidate for the Condorcet winner, and then choosing a second ranker that has the best chance of…”
Battle of Bandits (2018, background): “Several algorithms have been proposed for this problem which are designed to learn to play the best arm as often as possible over time [22, 4, 24, 25, 23, 16].”