A General Safety Framework for Learning-Based Control in Uncertain Robotic Systems

By Jaime Fernández Fisac, Anayo K. Akametalu, Melanie Nicole Zeilinger, Shahab Kaynama, Jeremy H. Gillula, Claire Jennifer Tomlin. IEEE Transactions on Automatic Control, 2017

580 citations recorded by the source corpus.

The proven efficacy of learning-based control schemes strongly motivates their application to robotic systems operating in the physical world. However, guaranteeing correct operation during the learning process is currently an unresolved issue, which is of vital importance in safety-critical systems. We propose a general safety framework based on Hamilton–Jacobi reachability methods that can work in conjunction with an arbitrary learning algorithm. The method exploits approximate knowledge of the system dynamics to guarantee constraint satisfaction while minimally interfering with the learning process. We further introduce a Bayesian mechanism that refines the safety analysis as the system acquires new evidence, reducing initial conservativeness when appropriate while strengthening guarantees through real-time validation. The result is a least-restrictive, safety-preserving control law that intervenes only when the computed safety guarantees require it, or confidence in the computed guarantees decays in light of new observations. We prove theoretical safety guarantees combining probabilistic and worst-case analysis and demonstrate the proposed framework experimentally on a quadrotor vehicle. Even though safety analysis is based on a simple point-mass model, the quadrotor successfully arrives at a suitable controller by policy-gradient reinforcement learning without ever crashing, and safely retracts away from a strong external disturbance introduced during flight.

Subfields: Control and Systems Engineering, Epidemiology

View Paper: https://www.semanticscholar.org/paper/171bfa2abddd30ad177cd620c86b7f8fa64964d1

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Control and Systems Engineering

1. Sergey Levine
2. Chelsea Finn
3. Suraj Nair
4. Pieter Abbeel
5. Julian Ibarz
6. Anca Dragan
7. Jie Tan
8. Joseph J. Lim

In Epidemiology

1. Sergey Levine
2. Chelsea Finn

Citations in the corpus

• Hamilton-Jacobi reachability: A brief overview and recent advances (2017, methodology): “In [56], [57], a general safety framework is proposed based on HJ reachability methods that can work in conjunction with an arbitrary learning algorithm.”
• A Review of Safe Reinforcement Learning: Methods, Theory and Applications (2022, methodology): “Moreover, some safe RL methods are proposed from the perspective of Model Predictive Control (MPC) [84], e.”
• PAC Confidence Predictions for Deep Neural Network Classifiers (2020, background): “There has also been work on safe learning-based control (Akametalu et al., 2014; Fisac et al., 2019; Bastani, 2019; Li & Bastani, 2020; Wabersich & Zeilinger, 2018; Alshiekh et al., 2018); however, these approaches are not applicable to perception-based control.”
• Learning to Synthesize Programs as Interpretable and Generalizable Policies (2021, background): “[72] Jaime F Fisac, Anayo K Akametalu, Melanie N Zeilinger, Shahab Kaynama, Jeremy Gillula, and Claire J Tomlin.”
• Measuring and avoiding side effects using relative reachability (2018, background): “A similar argument applies to reachability analysis methods (Mitchell et al., 2005; Gillula and Tomlin, 2012; Fisac et al., 2017), which require that a safe region must be reachable by a known conservative policy: the agent would be penalized if another agent or an environment event make the safe…”
• Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning (2021, methodology): “…122, 123, 124, 125, 128, 129, 130, 133, 134, 135, 136, 139, 140, 142), Parameters (126, 127, 141) Stabilization (128, 136, 139, 141, 142), Tracking (119, 120, 123, 124, 125, 126, 127, 129, 130, 131, 133, 134, 140), Navigation (121, 135), Loco-motion (122) S a f e L e a r n i n g i n R o b o t i c s”
• Penalizing Side Effects using Stepwise Relative Reachability (2018, methodology): “…methods: methods that preserve reversibility (the reachability of a starting state) [Moldovan and Abbeel, 2012; Eysenbach et al., 2017], and reachability analysis methods that require reachability of a safe region [Mitchell et al., 2005; Gillula and Tomlin, 2012; Fisac et al., 2017].”
• A Review of Safe Reinforcement Learning: Methods, Theories, and Applications (2024, unknown): “Inthis section,wewillinvestigatemodel-basedsafeRLmethodsfrom differentperspectives:policyoptimization-basedapproaches, controltheory-basedapproaches,formalmethods-basedap-proaches, and Gaussian process-based approaches.”
• Data-Enabled Predictive Control: In the Shallows of the DeePC (2018, unknown): “learning approaches [28], [29].”
• Policy Gradient Bayesian Robust Optimization for Imitation Learning (2021, background): “…to a single task rewards (Heger, 1994; Shen et al., 2014; Tamar et al., 2014; Tang et al., 2019), establish convergence to safe regions of the MDP (Thananjeyan et al., 2020b;a), or optimize a policy to avoid constraint violations (Achiam et al., 2017; Fisac et al., 2018; Thananjeyan et al., 2021).”
• Recovery RL: Safe Reinforcement Learning With Learned Recovery Zones (2020, methodology): “This idea has been explored in [4, 11], which utilize Hamilton-Jacobi reachability analysis to define a task policy and safety controller, and in the context”
• Learning to Walk in the Real World with Minimal Human Effort (2020, background): “Many researchers have proposed extensions of the existing deep RL algorithms to address safety, such as learning an additional safety layer that projects raw actions to a safe feasibility set [3, 25, 20, 17], learning the boundary of the safe states with a classifier [38, 49, 23], expanding the iden”