Lyapunov Design for Safe Reinforcement Learning

By Theodore J. Perkins, Andrew Barto. 2003

229 citations recorded by the source corpus.

Subfields: Reinforcement Learning, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

In Safety, Robustness & Interpretability

Citations in the corpus

Uncertainty-Aware Reinforcement Learning for Collision Avoidance (2017, background): “Methods based on reachability provide appealing theoretical guarantees, but cannot cope with rich sensory input and are often difﬁcult to scale to high-dimensional systems [23, 17, 10].”
Safe Model-based Reinforcement Learning with Stability Guarantees (2017, background): “In [20, 21], a priori known, safe global backup policies are used, while [22] learns to switch between several safe policies.”
A General Safety Framework for Learning-Based Control in Uncertain Robotic Systems (2017, background): “Lyapunov-based reinforcement learning [10] allowed a learning agent to switch between a number of precomputed”
Handbook of Learning and Approximate Dynamic Programming (IEEE Press Series on Computational Intelligence) (2004, background): “[28], [ 40 ]). By creating and using subgoals, the agent is able to accelerate its learning on the current task, and to transfer its expertise to other related tasks.”
Safe Reinforcement Learning (2015, background): “However, if information about the transition function is known, then control theoretic approaches that leverage this knowledge may require less historical data to find good policies while enforcing various safety guarantees (Akametalu et al., 2014, Kretchmar et al., 2001, Perkins and Barto, 2003).”
Reducing Conservativeness in Safety Guarantees by Learning Disturbances Online: Iterated Guaranteed Safe Online Learning (2012, methodology): “In a related manner, Perkins and Barto [12] have successfully used Lyapunov design methods to guarantee safety for high-level controllers which learn how to switch between a number of base-level controllers.”
On Ensuring that Intelligent Machines Are Well-Behaved (2017, background): “…ensure (sometimes with high probability, rather than surely) that a system will never enter a predefined unsafe set of states [Tomlin, 1998; Oishi et al., 2001; Perkins and Barto, 2003; Mitchell et al., 2005; Hans et al., 2008; Arvelo and Martins, 2012; Akametalu et al., 2014; Zilberstein, 2015].”
A Review of Safe Reinforcement Learning: Methods, Theory and Applications (2022, methodology): “To achieve safety policy, a growing number of approaches have been developed over the last few decades, such as primal-dual methods [51, 152, 160, 207, 235, 239, 271, 301], trust region with safety constraints methods [4, 32, 33, 106, 283, 307], formal methods [15, 23, 25, 86, 87, 120, 189, 208, 229”
A Lyapunov-based Approach to Safe Reinforcement Learning (2018, methodology): “Using Lyapunov functions in RL was first studied by [31], where Lyapunov functions were used to guarantee closed-loop stability of an agent.”
Preventing undesirable behavior of intelligent machines (2019, background): “For example, control theoretic algorithms have been developed that ensure (sometimes with high probability, rather than surely) that a system will never enter a predefined unsafe set of states [57, 58, 59, 60, 61, 62, 63, 64].”
A Review of Safe Reinforcement Learning: Methods, Theories, and Applications (2024, unknown): “Therefore, for the safe RL application research, we introduce some safe RL applications in this review, e.g., autonomous driving [56], [57], [58], [59], [60], [61], [62], [63], robotics [64], [65], [66], [67], [68], [69], video compression [70], vehicle schedule [1], [71], etc.”
Lyapunov-based Safe Policy Optimization for Continuous Control (2019, unknown): “L G ] 11 F e b 2019 Barto, 2002; Faust et al., 2014).”
Proximal Reinforcement Learning: A New Theory of Sequential Decision Making in Primal-Dual Spaces (2014, background): “Researchers have addressed safety concerns like these before [87].”
Recent Advances in Hierarchical Reinforcement Learning (2003, background): “Perkins and Barto [51] [50], for example, consider collections of options each of which descends on a Lyapunov function.”