The Epoch-Greedy algorithm for contextual multi-armed bandits

By J. Langford, Tong Zhang. 2007

480 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Contextual Dueling Bandits (2015, methodology): “To address this difficulty, we propose an approach used previously in other works on contextual bandits (Langford and Zhang, 2007; Dudı́k et al., 2011; Agarwal et al., 2014).”
Survey on Automated Machine Learning (2019, background): “In the literature many different policies have been proposed, for example greedy and -greedy methods (Langford and Zhang, 2008) or the upper confidence bound (Auer, 2002).”
Doubly Robust Policy Evaluation and Optimization (2014, background): “Contextual bandits (Auer et al., 2002/03; Langford and Zhang, 2008), sometimes known as associative reinforcement learning (Barto and Anandan, 1985), are a natural generalization of the classic multiarmed ban-”
A contextual-bandit approach to personalized news article recommendation (2010, methodology): “Following previous work [18], we call it a contextual bandit.”
Unbiased offline evaluation of contextual-bandit-based news article recommendation algorithms (2010, methodology): “Another general contextual bandit algorithm is the epoch-greedy algorithm [18] that is similar to ǫ -greedy with adaptively shrinking ǫ .”
The offset tree for learning with partial labels (2008, methodology): “Sample complexity results for policy evaluation in reinforcement learning [9] and contextual bandits [13] show that Empirical Risk Minimization type algorithms can find a good policy in a non-interactive setting.”
Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems (2012, neutral): “The general stochastic model of Section 4.3 for contextual bandits with finitely many arms is due to Langford and Zhang [2007].”
Taming the Monster: A Fast and Simple Algorithm for Contextual Bandits (2014, methodology): “…in an i.i.d. setting (formally deﬁned in Section 2.1), it is possible to create ǫ -greedy (Sutton and Barto, 1998) or epoch-greedy (Langford and Zhang, 2007) algorithms that run in time O (log | Π | ) with only a single call to the oracle per round. regret However, these algorithms have…”
Thompson Sampling for Contextual Bandits with Linear Payoffs (2012, background): “The name contextual bandits was coined in Langford & Zhang (2007).”
Interactively optimizing information retrieval systems as a dueling bandits problem (2009, methodology): “Related on-line methods use absolute measures of individual retrieved results (Pandey et al., 2007; Langford & Zhang, 2007; Radlinski et al., 2008a).”
The Unfair Externalities of Exploration (2017, background): “We consider contextual bandits , a standard model of the explore-exploit tradeoﬀ for content optimization scenarios (e.g., [1, 12, 13]).”
PHYRE: A New Benchmark for Physical Reasoning (2019, methodology): “Because the agent can only perform a single action to solve a PHYRE task, PHYRE is similar to a contextual bandit setting [23, 25].”
Contextual Bandits with Linear Payoff Functions (2011, methodology): “While experiments show LinUCB is probably sufficient in practice Li et al. [2010], there is technical difficulty in analyzing it.”
Stochastic Structured Prediction under Bandit Feedback (2016, background): “Contextual one-state MDPs are also known as contextual bandits [11, 13] which operate in a scenario of maximizing the expected reward for selecting an arm of a multi-armed slot machine.”
Introduction to Multi-Armed Bandits (2019, background): “Applications to the Web trace back to (Pandey et al., 2007a,b; Langford and Zhang, 2007) for ad placement, (Li et al.”