Autonomous reinforcement learning on raw visual input data in a real world application

By Sascha Lange, Martin A. Riedmiller, Arne Voigtländer. 2012

249 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Management Science and Operations Research, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

In Reinforcement Learning

Citations in the corpus

End-to-End Training of Deep Visuomotor Policies (2015, background): “More recent work on sensorimotor deep learning has tackled simple task-space motion [17] and used unsupervised learning to obtain low-dimensional state spaces from images [34], but such methods are limited to tasks with a low-dimensional structure.”
From Pixels to Torques: Policy Learning with Deep Dynamical Models (2015, methodology): “on images: For instance, (Mnih et al., 2015) present an approach based on Deep-Q-learning, in which human-level game strategies are learned autonomously, purely based on pixel information. Moreover, (Lange et al., 2012) presented an approach that learns good discrete actions to control a slot car ba”
Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates (2016, background): “Although there has been considerable research on reinforcement learning with general-purpose neural networks for some time [13], [14], [15], [16], [17], such methods have only recently been developed to the point where they could be applied to continuous control of highdimensional systems, such as 7”
Deep learning in neural networks: An overview (2014, methodology): “t al., 2010; Cuccu et al., 2011), just like SL may proﬁt from UL, e.g., Sec. 5.8, 5.10, 5.15. For exam ple, NFQ (Riedmiller, 2005) was applied to real-world control tasks (Lange and Riedmiller, 2010; Riedmiller et al., 2012) where purely visual inputs were compactly encoded by deep autoencoders (Sec”
Unsupervised state representation learning with robotic priors: a robustness benchmark (2017, methodology): “This is an alternative approach to selecting a state representation from a set ([7], [8]), or creating an autoencoder to compress information into a lower dimensional state ([9], [10]), [11].”
A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation (2016, positive): “ original data and improve the efﬁciency, multilayer neural networks were used to train the auto-encoder model (Lange and JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 9 Riedmiller, 2010; Lange et al., 2012; Finn et al., 2015). Lange and Riedmiller (2010) deﬁned the deep-ﬁtted Q algorith”
Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning (2017, background): “Model-based algorithms are known in general to outperform model-free learners in terms of sample complexity [6], and in practice have been applied successfully to control simple robotic systems both in simulation and in the real world [5, 39, 18].”
Deep visual foresight for planning robot motion (2016, methodology): “Other works have used model-based approaches with a learned low-dimensional embedding of images [28], [29], [30], [31], defining the objective using an image of the goal; however, such methods have been demonstrated either on simple synthetic images, or without generalization to new objects.”
Deep spatial autoencoders for visuomotor learning (2015, methodology): “[1] use autoencoders to acquire a state space for Q-learning and evaluate on simple real-world problems with 2 to 3 dimensions.”
Learning to Poke by Poking: Experiential Learning of Intuitive Physics (2016, methodology): “A number of recent methods have also been proposed for learning representations for vision-based control using autoencoders to transform visual inputs into a low-dimensional state space (Lange et al., 2012; Finn et al., 2015; Lange & Riedmiller, 2010; Kietzmann & Riedmiller, 2009).”
Unsupervised Learning for Physical Interaction through Video Prediction (2016, methodology): “In the domain of reinforcement learning, model-based methods have been proposed that learn prediction on images [14, 27], but they have either used synthetic images or instance-level models, and have not demonstrated”
Learning Visual Predictive Models of Physics for Playing Billiards (2015, background): “Another body of work has sought to learn useful features from raw sensory observations using methods such as deep autoencoders, in the hopes that these unsupervised learning techniques would produce a low-dimensional space that is better suited for reinforcement learning and control (Kietzmann & Rie”
Visual Reinforcement Learning with Imagined Goals (2018, methodology): “s/. 2 Related Work While prior works on vision-based deep reinforcement learning for robotics can efﬁciently learn a variety of behaviors such as grasping [33, 32, 24], pushing [1, 8, 10], navigation [30, 21], and other manipulation tasks [26, 23, 30], they each make assumptions that limit their app”
Learning Latent Plans from Play (2019, background): “Learning powerful goalconditioned policies with reinforcement learning can produce policies with good long-horizon performance, but is difficult in terms of both the number of samples required and the need for extensive on-policy exploration [43, 42, 31, 12, 14, 29, 33, 30].”
On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, methodology): “For example, NFQ [201] was applied to real-world control tasks [138, 202] where purely visual inputs were compactly encoded in hidden layers of deep autoencoders [245, Sec.”
A Brief Survey of Deep Reinforcement Learning (2017, neutral): “NFQ was later extended to train a network to drive a slot car using raw visual inputs from a camera over the race track, by combining a deep autoencoder to reduce the dimensionality of the inputs with a separate branch to predict Q-values [69].”
Deep predictive policy training using reinforcement learning (2017, background): “[7] trained a deep convolutional autoencoder to learn a state space representation to autonomously control a toy race car.”
Deep reinforcement learning with successor features for navigation across similar environments (2016, background): “Learning state representations for RL via auto-encoders has previously been considered several times in the literature [26], [27], [28].”
One-shot learning of manipulation skills with online dynamics adaptation and neural network priors (2015, neutral): “One approach for addressing this is to learn a latent state representation from raw sensory input, as proposed in recent work [23], [24], [25], and perform MPC on”