Scene Memory Transformer for Embodied Agents in Long-Horizon Tasks
By Kuan Fang, Alexander Toshev, Li Fei-Fei, Silvio Savarese. 2019
232 citations recorded by the source corpus.
Many robotic applications require the agent to perform long-horizon tasks in partially observable environments. In such applications, decision making at any step can depend on observations received far in the past. Hence, being able to properly memorize and utilize the long-term history is crucial. In this work, we propose a novel memory-based policy, named Scene Memory Transformer (SMT). The proposed policy embeds and adds each observation to a memory and uses the attention mechanism to exploit spatio-temporal dependencies. This model is generic and can be efficiently trained with reinforcement learning over long episodes. On a range of visual navigation tasks, SMT demonstrates superior performance to existing reactive and memory-based policies by a margin.
Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, LLM Agents & Tool Use, Reinforcement Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Aerospace Engineering
- Sergey Levine
- Abhinav Kumar Gupta
- Kristen Grauman
- Dhruv Batra
- Saurabh Gupta
- Jitendra Malik
- Trevor Darrell
- Chelsea Finn
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Trevor Darrell
- Jitendra Malik
- Dacheng Tao
- Kristen Grauman
- Xiaogang Wang
- Abhinav Kumar Gupta
- Li Fei-Fei
In LLM Agents & Tool Use
- Yi Wu
- Igor Mordatch
- Peter Alexander Jansen
- Kristen Grauman
- Tushar Nagarajan
- Dhruv Batra
- Stuart J. Russell
- Devi Parikh
In Reinforcement Learning
- Sergey Levine
- Chelsea Finn
- Pieter Abbeel
- Abhinav Kumar Gupta
- Ruslan Salakhutdinov
- Karol Hausman
- Igor Mordatch
- Ofir Nachum
Citations in the corpus
- Teaching Autoregressive Language Models Complex Tasks By Demonstration (2021, background): “They have also shown great promise on non-linguistic tasks such as image recognition [5], image generation [6], navigation [7], and symbolic manipulation [8].”
- A Survey of Embodied AI: From Simulators to Research Tasks (2021, methodology): “Another recent work is a scene memory transformer which uses the self-attention mechanism adapted from the Transformer model [68] over the scene memory in its policy network [56].”
- An Exploration of Embodied Visual Exploration (2020, unknown): “More generally, this idea can be exploited for learning to visit other things of interest such as objects (similar to the search task from [18]) and landmarks, as we define below.”
- Semantic Curiosity for Active Visual Learning (2020, unknown): “The difference between these approaches and our method is twofold: first, we train semanticallyaware exploration policies as compared spatial coverage maximization in some prior works [9, 14], and second, we train our policy in an unsupervised fashion, without requiring any ground truth labels from ”
- Shaping embodied agent behavior with activity-context priors from egocentric video (2021, methodology): “We use object coverage [17, 49], which rewards the agent for visiting (but not interacting with) new objects.”
- History Aware Multimodal Transformer for Vision-and-Language Navigation (2021, background): “Such condensed states might be sub-optimal for capturing essential information in extended trajectories [17].”
- RT-1: Robotics Transformer for Real-World Control at Scale (2022, background): “, 2022a) as well as simulated robot navigation (Fang et al., 2019), locomotion (Janner et al.”
- Open-vocabulary Queryable Scene Representations for Real World Planning (2022, background): “A few of these algorithms construct a semantic map of the current region before planning in that region [1], [22]–[24].”
- Ego-Topo: Environment Affordances From Egocentric Video (2020, background): “Instead, recent work in embodied visual navigation explores learned maps that leverage both visual patterns as well as geometry, with the advantage of extrapolating to novel environments ( e.g ., [23, 22, 60, 26, 10]).”
- Learning Object Relation Graph and Tentative Policy for Visual Navigation (2020, methodology): “The methods [6,9] adopt a collision reward and collision detector to avoid collisions.”
- Bayesian Relational Memory for Semantic Visual Navigation (2019, background): “[18] also consider extracting visual concepts dynamically by treating every received input frame as an individual concept and storing them in the memory.”
- Occupancy Anticipation for Efficient Exploration and Navigation (2020, background): “Prior work on exploration [11,17,10,52] often uses area-coverage—the area seen in the environment during navigation—as a reward function to guide exploration.”
- Learning Affordance Landscapes for Interaction Exploration in 3D Environments (2020, background): “• OBJCOVERAGE [18, 47] rewards an agent for visiting new objects (moving close to it, and centering it in view), but not for interacting with them.”
- Learning to Explore using Active Neural SLAM (2020, methodology): “Both Gibson and Matterport datasets are based on real-world scene reconstructions are thus significantly more realistic than synthetic SUNCG dataset (Song et al., 2017) used for past research on exploration (Chen et al., 2019; Fang et al., 2019).”
- Learning to Visually Navigate in Photorealistic Environments Without any Supervision (2020, unknown): “There has been a recent interest in using techniques from deep learning in the context of SLAM, or more generally, in the context of navigation [2, 11, 13, 15, 16, 22, 23, 32, 36].”
- Neural Topological SLAM for Visual Navigation (2020, background): “This can be in the form of vanilla neural network memories such as LSTMs [25, 29], or transformers [14].”
- Seeing the Un-Scene: Learning Amodal Semantic Maps for Room Navigation (2020, background): “With the motivation of generalizing to novel environments and learning semantic cues, a number of end-to-end learning based approaches have been developed in the recent past [14,17,28,31,47].”
- Spatio-Temporal Graph Transformer Networks for Pedestrian Trajectory Prediction (2020, background): “, stock prediction [30], robot decision making [12] etc.”