Learning Transferable Policies for Monocular Reactive MAV Control

By Shreyansh Daftry, J. Bagnell, Martial Hebert. 2016

89 citations recorded by the source corpus.

The ability to transfer knowledge gained in previous tasks into new contexts is one of the most important mechanisms of human learning. Despite this, adapting autonomous behavior to be reused in partially similar settings is still an open problem in current robotics research. In this paper, we take a small step in this direction and propose a generic framework for learning transferable motion policies. Our goal is to solve a learning problem in a target domain by utilizing the training data in a different but related source domain. We present this in the context of an autonomous MAV flight using monocular reactive control, and demonstrate the efficacy of our proposed approach through extensive real-world flight experiments in outdoor cluttered environments.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

MagNet: A Two-Pronged Defense against Adversarial Examples (2017, background): “They are used in autonomous control for robots and vehicles [1, 3, 4], financial systems [32], medical treatments [31], information security [12, 29], and human-computer interaction [11, 13].”
DARLA: Improving Zero-Shot Transfer in Reinforcement Learning (2017, background): “, 2016) has not yet translated as well to reality, despite various attempts (Tobin et al., 2017; Tzeng et al., 2016; Daftry et al., 2016; Finn et al., 2015; Rusu et al., 2016).”
Sim-to-Real Transfer of Robotic Control with Dynamics Randomization (2017, background): “Daftry et al. [14] applied a similar approach to transfer policies for controlling aerial vehicles to different environments and vehicle models.”
Cognitive Mapping and Planning for Visual Navigation (2017, methodology): “In context of navigation, learning and DRL has been used to obtain policies [4,12,13,21,35,50,56,59,64].”
Learning Invariant Feature Spaces to Transfer Skills with Reinforcement Learning (2017, background): “kly together than separately. More recent work in deep learning has also looked at transferring policies by reusing policy parameters between environments (Rusu et al., 2016a;b; Braylan et al., 2015; Daftry et al., 2016), using either regularization or novel neural network architectures, though this”
Behavioral Cloning from Observation (2018, methodology): “Like BC techniques, IRL methods usually assume that state-action pairs are available [Finn et al. , 2016; Ho and Ermon, 2016; Ho et al. , 2016], and also that the reward is a function of both states and actions.”
Adapting Deep Visuomotor Representations with Weak Pairwise Constraints (2015, neutral): “Contemporaneously to our work, [37] also reported success with a domain confusion-style regularizer on a domain adaptive visual behavior task on autonomous MAV flight.”
End-to-End Learning of Driving Models from Large-Scale Video Datasets (2016, background): “, as was done in [17], and more recently in [23, 20, 3]).”
Robust Visual Domain Randomization for Reinforcement Learning. (2019, background): “…whereas the notion of learning similar state representations to improve transfer to new domains is not new Tzeng et al. (2015); Gupta et al. (2017); Daftry et al. (2016), previous work has focused on cases when simulated and target domains are both known, and cannot straightforwardly be applied…”
Unifying Map and Landmark Based Representations for Visual Navigation (2017, background): “Most such approaches [6, 11, 15, 26, 28] study task-agnostic collision avoidance, and don’t build representations for spaces or plan paths.”
Towards Adapting Deep Visuomotor Representations from Simulated to Real Environments (2015, result): “on of this paper [36], we demonstrated initial results using domain confusion constraints on PR2 visuomotor policies but without the pairwise constraint reported below. Contemporaneously to our work, [37] also reported success with a domain confusion-style regularizer on a domain adaptive visual beh”
Transferable Representation Learning in Vision-and-Language Navigation (2019, methodology): “ions are sampled from the model during training, and supervised using a shortest-path action to reach the goal. For the RCM agent, learning is performed in two separate phases, (1) behavioral cloning [4, 39, 8] and (2) REINFORCE policy gradient updates [41]. The agent’s policy is initialized using b”
Autonomous Cars: Research Results, Issues, and Future Challenges (2019, background): “Furthermore, this algorithm also addresses the limitation of traditional end-to-end learning that is possible only on a specific dataset as in [67].”
Memory Augmented Control Networks (2017, background): “Purely convolutional architectures (CNNs) perform poorly when applied to planning problems due to the reactive nature of the policies learned by them [5],[6],[7].”
Imitation Learning: A Survey of Learning Methods, Environments and Metrics (2024, unknown): “In robotics, most domain-specific metrics measure time [28, 59, 63] or failure rate [10, 28].”
Efficiently Guiding Imitation Learning Algorithms with Human Gaze (2020, background): “Behavior Cloning (Bain and Sommut 1999; Ross, Gordon, and Bagnell 2011; Daftry, Bagnell, and Hebert 2016) is a standard approach used for imitation learning, where an agent trains a classifier or regressor to replicate an expert’s policy by using the demonstrated states and actions as input data…”
Stay on the Path: Instruction Fidelity in Vision-and-Language Navigation (2019, methodology): “Training is performed using two separate phases, (1) behavioral cloning (Bain and Sammut, 1999; Wang et al., 2019; Daftry et al., 2016) and (2) REINFORCE policy gradient updates (Williams, 1992).”