Learning Visual Feature Spaces for Robotic Manipulation with Deep Spatial Autoencoders

By Chelsea Finn, Xin Yu Tan, Yan Duan, Trevor Darrell, Sergey Levine, Pieter Abbeel. 2015

81 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Reinforcement Learning

Citations in the corpus

End-to-End Training of Deep Visuomotor Policies (2015, methodology): “A promising direction for addressing this limitation is to combine our method with unsupervised state-space learning, as proposed in several recent works, including our own (Lange et al., 2012; Watter et al., 2015; Finn et al., 2015).”
Time-Contrastive Networks: Self-Supervised Learning from Video (2017, methodology): “It consists of the Inception model up until the layer “Mixed 5d” (initialized with ImageNet pre-trained weights), followed by 2 convolutional layers, a spatial softmax layer [42] and a fully-connected layer.”
Learning to Optimize (2016, methodology): “We use an off-the-shelf reinforcement learning algorithm known as guided policy search [17], which has demonstrated success in a variety of robotic control settings [18, 10, 19, 12].”
SE3-nets: Learning rigid body motion using deep neural networks (2016, methodology): “Recently, deep models have been used for learning dynamics models in robotics and reinforcement learning, by mapping raw pixel images to low-dimensional (latent) encodings on top of which standard optimal control methods are applied [13], [14].”
Learning robot in-hand manipulation with tactile features (2015, background): “In contrast to earlier work on learning representations for RL [10, 17, 18], we aim to learn feedback policies directly from the high-dimensional space in an iterative on-policy fashion, allowing the improving policy to generate more relevant samples for the reinforcement learner.”
Spatial Broadcast Decoder: A Simple Architecture for Learning Disentangled Representations in VAEs (2019, background): “Other types of generative models already use convolutional latent distributions [Finn et al., 2015, Levine et al., 2015, Eslami et al., 2018], unlike the ﬂat vectors we consider here.”
3D Simulation for Robot Arm Control with Deep Q-Learning (2016, neutral): “However, our approach differs in that we do not require any real-world training, and attempt to directly apply policies learned in simulation, over to a real robot.”
AlphaPose: Whole-Body Regional Multi-Person Pose Estimation and Tracking in Real-Time (2022, background): “Several previous works have studied the softargmax operation to read continuous joint locations from heatmaps [41], [42], [43], [44], [45], [46].”
Human Pose Regression by Combining Indirect Part Detection and Contextual Information (2017, unknown): “All the previous methods that are based on detection need additional steps on training to produce artiﬁcial ground truth from joint positions, which represent an additional processing stage and additional hype-parameters, since the ground truth heat maps have to be deﬁned by hand.”
Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learning (2021, background): “…have signiﬁcantly bridged the gap between state-based and image-based RL. Future prediction objectives [Hafner et al., 2018, 2019, Yan et al., 2020, Finn et al., 2015, Pinto et al., 2016, Agrawal et al., 2016] and other auxiliary objectives [Jaderberg et al., 2016, Zhan et al., 2020, Young et…”
Learning state representation for deep actor-critic control (2016, methodology): “In [20], [18] this assumption is used to learn a mapping between visual observations and a state representation that gradually changes over time.”
Improving Sample Efficiency in Model-Free Reinforcement Learning from Images (2019, methodology): “Following prior work (Lange and Riedmiller 2010; Finn et al. 2015) we explore unsupervised pretraining via an image-based autoencoder (AE).”
Decoupling feature extraction from policy learning: assessing benefits of state representation learning in goal based robotics (2018, methodology): “, 2017), and using both data-reconstruction and priors loss functions (Finn et al., 2015).”
Learning Actionable Representations with Goal-Conditioned Policies (2018, methodology): “…to model the data distribution, and the latent variables are then used as a representation (Rasmus et al., 2015; Dumoulin et al., 2016; Kingma & Welling, 2013; Ghadirzadeh et al., 2017b; Finn et al., 2015; Ghadirzadeh et al., 2017a; Curran et al., 2015; Goroshin et al., 2015; Higgins et al., 2017).”
The Surprising Effectiveness of Linear Models for Visual Foresight in Object Pile Manipulation (2020, methodology): “ns. image gets feedbacked, making the closed-loop system get stuck in a loop. This failure mode agrees with the observation in [25], where policies trained in the latent-space of spatial autoencoders [8] repeatedly produced actions that did not change the scene. We observed these failure cases and f”
Cooperation and communication in multiagent deep reinforcement learning (2016, background): “However, there are a few researchers who have made notable headway learning on physical robots (Levine et al., 2015; Finn et al., 2015; Finn and Levine, 2016), employing trajectory optimization to reduce the amount of data that needs to be collect on the robot or many robots that can collect data in”
Time-Contrastive Networks: Self-Supervised Learning from Multi-view Observation (2017, methodology): “It consists of the Inception model up until the layer “Mixed 5d” (initialized with ImageNet pre-trained weights), followed by 2 convolutional layers, a spatial softmax layer [9] and a fullyconnected layer.”
Learning Semantic Embedding Spaces for Slicing Vegetables (2019, methodology): “Previous works in robotics have looked into learning visual representations for various tasks [5], [6], [7].”
Graph-Structured Visual Imitation (2019, background): “Numerous objectives have been proposed to learn full image or image sequence convolutional image embeddings, such as multiview invariant and time-contrastive objectives in [2, 20], forward and inverse dynamics model learning in [4, 21], or reconstruction and temporal prediction objectives in [22, 23”
Learning Actionable Representations from Visual Observations (2018, methodology): “ween the original TCN [3] and mfTCN. First, there is a 3D convolution layer in mfTCN that takes convolutional feature maps from different time steps as input. Second, we do not have a spatial softmax [37] layer in mfTCN which was used by the original TCN to reduce dimensionality. B. Learning Control”