Mid-Level Visual Representations Improve Generalization and Sample Efficiency for Learning Visuomotor Policies
By Alexander Sax, Bradley Emi, Amir Zamir, Leonidas J. Guibas, Silvio Savarese, Jitendra Malik. 2018
74 citations recorded by the source corpus.
Subfields: Reinforcement Learning, Representation & Self-Supervised Learning, Robotics & Control
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Reinforcement Learning
- Silvio Savarese
- Sergey Levine
- Kristen Grauman
- Li Fei-Fei
- Yuke Zhu
- Jitendra Malik
- Chelsea Finn
- Amy Zhang
In Representation & Self-Supervised Learning
- Jitendra Malik
- Amir Zamir
- Leonidas J. Guibas
- Alexander Sax
- Silvio Savarese
- Kristen Grauman
- Li Fei-Fei
- Sergey Levine
In Robotics & Control
- Silvio Savarese
- Li Fei-Fei
- Yuke Zhu
- Jitendra Malik
- Kristen Grauman
- Linxi (Jim) Fan
- Amir Zamir
- Leonidas J. Guibas
Citations in the corpus
- Taskonomy: Disentangling Task Transfer Learning (2018, methodology): “We systematically study this question in [Sax et al., 2018], by integrating a generic perceptual skill set based on TaskonPolicy”
- VIMA: General Robot Manipulation with Multimodal Prompts (2022, background): “This is consistent with findings from Sax et al. (2018), which demonstrates that embodied agents conditioned on mid-level visual representations tend to be significantly more sample-efficient than end-to-end control from raw pixels.”
- Occupancy Anticipation for Efficient Exploration and Navigation (2020, background): “More recent work uses deep learning to learn navigation [79,20,56,41,77,75,59,64] or exploration [48,6,57,28,51] policies in an end-to-end fashion.”
- Bisimulation Makes Analogies in Goal-Conditioned Reinforcement Learning (2022, background): “…and Brock, 2015; Jonschkowski et al., 2017), via data augmentations and contrastive learning (Laskin et al., 2020a), dynamics (Watter et al., 2015; Gelada et al., 2019), or proxy computer vision tasks (e.g., image reconstruction or segmentation) (Lange and Riedmiller, 2010; Sax et al., 2019).”
- MultiMAE: Multi-modal Multi-task Masked Autoencoders (2022, background): “This is supported by previous findings [58, 70, 99] showing that there is usually no single visual pre-training task that transfers well to any arbitrary other task, and instead, a set is required.”
- Auxiliary Tasks Speed Up Learning PointGoal Navigation (2020, unknown): “Another approach [34] transfers mid-level visual features from Taskonomy [33], freezing visual representation encoders.”