Ieee Transactions on Systems, Man and Cybernetics—part C: Applications and Reviews 1 a Survey of Actor-critic Reinforcement Learning: Standard and Natural Policy Gradients

By Ivo Grondman, Lucian Busoniu, G. D. Lopes, Robert Babuška

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Reinforcement Learning in Healthcare: A Survey (2019, background): “There has been a significant achievement in the continuous control using AC [74] and PS methods [42] in the past few years, particularly from the area of robotic control and DRL.”
Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning (2017, methodology): “are a good choice for model-free fine-tuning since they do not require any critic or value function for initialization [41], though our method could also be combined with other modelfree RL algorithms.”
An Off-policy Policy Gradient Theorem Using Emphatic Weightings (2018, background): “…[Witten, 1977, Barto et al., 1983] and many techniques since proposed to reduce variance of the estimates of this gradient [Konda and Tsitsiklis, 2000, Weaver and Tao, 2001, Greensmith et al., 2004, Peters et al., 2005, Bhatnagar et al., 2008, 2009, Grondman et al., 2012, Gu et al., 2016].”
On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, methodology): “1, except that C is now an FNN or RNN trained by evolutionary algorithms [200, 255, 105, 56, 68] applied to NNs [165, 321, 180, 259, 72, 90, 89, 110, 94], or by policy gradient methods [314, 315, 316, 274, 18, 1, 63, 128, 313, 210, 192, 191, 256, 85, 312, 190, 82, 93][245, Sec.”
Learning to Learn: Meta-Critic Networks for Sample Efficient Learning (2017, methodology): “Conventional critic networks [3, 13] are conditioned on a given task and actor that they are trained to criticise.”
Partial Policy-Based Reinforcement Learning for Anatomical Landmark Localization in 3D Medical Images (2018, methodology): “tion as the critic to reduce the variance [11].”
Human-Like Autonomous Car-Following Model with Deep Reinforcement Learning (2018, methodology): “The updated value function is then used to improve the actor’s policy (Grondman et al., 2012).”
FIRM: An Intelligent Fine-Grained Resource Management Framework for SLO-Oriented Microservices (2020, background): “8), and that is suitable for continuous stochastic environments, converges faster, and has lower variance [41].”
Adaptive Dynamic Programming for Control: A Survey and Recent Advances (2021, methodology): “An improved learning algorithm using policy gradient in policy improvement was adopted in [90] and [91].”
VPE: Variational Policy Embedding for Transfer Reinforcement Learning (2018, methodology): “We identify the policy parameters φ by maximizing the master Q-function similar to actor-critic learning [27],”
Actor-critic reinforcement learning for tracking control in robotics (2016, background): “For continuous state and action spaces, such as in the case of a manipulator arm, the actor and critic need to be approximated [11].”
Knowledge Tracing: A Survey (2022, methodology): “The authors used a DKT model [89] to trace the progress in the knowledge state of a student after taking a specific course module, and then, an actor-critic reinforcement learning agent [42] considers the knowledge state across different course modules and their predefined relationships (i.”
Robust Actor-Critic Contextual Bandit for Mobile Health (mHealth) Interventions (2018, background): “(b) From the perspective of optimization, the actor-critic algorithm has great properties of quick convergence with low variance [10].”
Application of Machine Learning in Wireless Networks: Key Techniques and Open Issues (2018, background): “In addition, since value function and policy are updated separately, policy knowledge transfer is easier to achieve [141].”