Algorithms for Reinforcement Learning

By Csaba Szepesvari. 2010

1,409 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Control and Systems Engineering, Management Science and Operations Research, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Control and Systems Engineering

In Management Science and Operations Research

In Reinforcement Learning

Citations in the corpus

Taming the Noise in Reinforcement Learning via Soft Updates (2015, methodology): “Such awareness to the cost of exploration is usually attributed to on-policy algorithms, such as SARSA [2, 4] and Expected-SARSA [15, 16].”
Neural Approaches to Conversational AI (2018, neutral): “er chapters. For a comprehensive survey, interested readers are referred to excellent textbooks and reviews, such as Sutton and Barto (2018); Kaelbling et al. (1996); Bertsekas and Tsitsiklis (1996); Szepesvari (2010); Wiering and van Otterlo (2012); Li (2019).´ 2.3.1 Foundations Reinforcement learn”
Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning (2015, background): “Uncertainty is important in reinforcement learning (RL) as well (Szepesvári, 2010).”
Value-Decomposition Networks For Cooperative Multi-Agent Learning (2017, neutral): “Q ∗ is optimal (e.g. Szepesvári, 2010).”
Modelling transition dynamics in MDPs with RKHS embeddings (2012, methodology): “Q by π Q ( x ) := argmax a ∈A Q ( x, a ) (choosing arbitrarily in the case of a tie) and the optimal action-value function, (1) so that π ∗ = π Q ∗ (see e.g. (Szepesvari, 2009) for this background).”
Batch Policy Gradient Methods for Improving Neural Conversation Models (2017, methodology): “In the last step, we have used the fact that E[π(a|s)ψ(a|s)h(s)] = 0 for any function h : S → R of the current state (Szepesvári, 2010).”
Information-Theoretic Considerations in Batch Reinforcement Learning (2019, methodology): “A representative algorithm for this setting is Fitted Q-Iteration (FQI) (Ernst et al., 2005; Szepesvári, 2010).”
A General Family of Robust Stochastic Operators for Reinforcement Learning (2018, methodology): “Through iterative application of a convergent operator, value-based reinforcement learning (RL) generates successive refinements of an initial value function [13, 20, 19].”
Reinforcement learning (2019, positive): “Modern RL sits in the intersection of many diverse disciplines and we take an approach based on machine learning (Sutton and Barto, 2018; Szepesv´ari, 2010) to provide this introduction.”
Successor Features for Transfer in Reinforcement Learning (2016, methodology): “Therefore, in principle any RL method can be used to compute ψ [24].”
Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems (2012, neutral): “The interested reader is addressed to [108, 157, 158].”
An Algorithmic Perspective on Imitation Learning (2018, methodology): “For an overview of reinforcement learning methods, please refer to [Sutton and Barto, 1998, Szepesvari, 2010, Wiering and van Otterlo, 2012, Sugiyama et al., 2013] and for an overview in reinforcement learning in robotics, please refer to Kober et al. [2013], Deisenroth et al. [2013b].”
APRIL: Active Preference-learning based Reinforcement Learning (2012, neutral): “Reinforcement learning (RL) [26, 27] raises a main issue, that of the prior knowledge needed to eﬃciently converge toward a (nearly) optimal policy.”
On the Sample Complexity of Reinforcement Learning with a Generative Model (2012, methodology): “Therefore, we did not compare our results with the-state-of-the-art of PAC-MDP (Strehl et al., 2009; Szita & Szepesv´ari, 2010) and upper-conﬁdence bound based algorithms (Bartlett & Tewari, 2009; Jaksch et al., 2010), in which the choice of the exploration policy has an inﬂuence on the behavior of…”
Generalization and Exploration via Randomized Value Functions (2014, methodology): “Our experiments focus on learning over episodes of a finite-horizon Markov decision process and use a version of RLSVI designed for that task, but we also propose a version of RLSVI that addresses continual learning in an infinite-horizon discounted Markov decision process.”
Compress and Control (2014, methodology): “We also compared CNC to a first-visit Monte Carlo value estimate (Szepesvári 2010).”
Finite-Sample Analysis of Proximal Gradient TD Algorithms (2015, background): “However, it has been shown that the above update rules do not update the value function parameter θ in the gradient direction of NEU and MSPBE, and thus, NEU and MSPBE are not the true objective functions of the GTD and GTD2 algorithms [Szepesvári, 2010].”
A unified view of entropy-regularized Markov decision processes (2017, methodology): “The idea of entropy regularization has also been used extensively in the reinforcement learning literature (Sutton and Barto, 1998; Szepesvári, 2010).”