Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning

By Lukas Brunke, Melissa Greeff, Adam W. Hall, Zhaocong Yuan, Siqi Zhou, Jacopo Panerati, Angela P. Schoellig. Annu. Rev. Control. Robotics Auton. Syst., 2021

983 citations recorded by the source corpus.

The last half decade has seen a steep rise in the number of contributions on safe learning methods for real-world robotic deployments from both the control and reinforcement learning communities. This article provides a concise but holistic review of the recent advances made in using machine learning to achieve safe decision-making under uncertainties, with a focus on unifying the language and frameworks used in control theory and reinforcement learning research. It includes learning-based control approaches that safely improve performance by learning the uncertain dynamics, reinforcement learning approaches that encourage safety or robustness, and methods that can formally certify the safety of a learned control policy. As data- and learning-based robot control methods continue to gain traction, researchers must understand when and how to best leverage them in real-world scenarios where safety is imperative, such as when operating in close proximity to humans. We highlight some of the open challenges that will drive the field of robot learning in the coming years, and emphasize the need for realistic physics-based benchmarks to facilitate fair comparisons between control and reinforcement learning approaches. Expected final online publication date for the Annual Review of Control, Robotics, and Autonomous Systems, Volume 5 is May 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Subfields: Artificial Intelligence, Control and Systems Engineering, Software

View Paper: https://www.semanticscholar.org/paper/d6e783bce3b8e3ad082c2757235c34cb86c4e653

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yuke Zhu
2. Li Fei-Fei
3. Anima Anandkumar
4. Linxi (Jim) Fan
5. Jun Wang
6. Mohit Bansal
7. Guanzhi Wang
8. Yunfan Jiang

In Control and Systems Engineering

1. Yuke Zhu
2. Li Fei-Fei
3. Linxi (Jim) Fan
4. Anima Anandkumar
5. Charles Dawson
6. Chuchu Fan
7. Shangding Gu
8. Yali Du

Citations in the corpus

• A Review of Safe Reinforcement Learning: Methods, Theory and Applications (2022, background): “For example, [45] investigating safe RL from the perspective of control theory and robotics, and the survey defines the soft constraints, probabilistic constraints and hard constraints.”
• VIMA: General Robot Manipulation with Multimodal Prompts (2022, background): “…require different skills and task specification formats, such as instruction following (Stepputtis et al., 2020), one-shot imitation (Finn et al., 2017; Duan et al., 2017), rearrangement (Batra et al., 2020), constraint satisfaction (Brunke et al., 2021a), and reasoning (Shridhar et al., 2020).”
• A Review of Safe Reinforcement Learning: Methods, Theories, and Applications (2024, positive): “Several surveys have already investigated safe RL problems and methods, e.g., [83], [84], [85], [86].”
• Safe Control With Learned Certificates: A Survey of Neural Lyapunov, Barrier, and Contraction Methods for Robotics and Control (2022, background): “, by maximizing some reward) while respecting a number of constraints on the behavior of the controlled system [25].”