Visual Semantic Planning Using Deep Successor Representations

By Yuke Zhu, Daniel Gordon, Eric Kolve, Dieter Fox, Li Fei-Fei, Abhinav Kumar Gupta, Roozbeh Mottaghi, Ali Farhadi. 2017

145 citations recorded by the source corpus.

A crucial capability of real-world intelligent agents is their ability to plan a sequence of actions to achieve their goals in the visual world. In this work, we address the problem of visual semantic planning: the task of predicting a sequence of actions from visual observations that transform a dynamic environment from an initial state to a goal state. Doing so entails knowledge about objects and their affordances, as well as actions and their preconditions and effects. We propose learning these through interacting with a visual and dynamic environment. Our proposed solution involves bootstrapping reinforcement learning with imitation learning. To ensure cross task generalization, we develop a deep predictive model based on successor representations. Our experimental results show near optimal results across a wide range of tasks in the challenging THOR environment.

Subfields: Computer Vision and Pattern Recognition, Control and Systems Engineering, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

In Reinforcement Learning

Citations in the corpus

Do As I Can, Not As I Say: Grounding Language in Robotic Affordances (2022, methodology): “Machine learning has recently been applied to enable abstract task specification, allow general primitives, or relax constraints [67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78].”
ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks (2019, background): “Previous work in the original AI2-THOR environment investigated the task of visual semantic planning [19, 57].”
Embodied Question Answering (2017, background): “AI2-THOR [18], Matterport3D [31], Stanford 2D-3D-S [35]).”
A Survey of Embodied AI: From Simulators to Research Tasks (2021, background): “This is inspired by the attention mechanism in neural networks, and successor representations [89, 90] and features [91] in reinforcement learning.”
IQA: Visual Question Answering in Interactive Environments (2017, methodology): “The learned baseline that we compare to (A3C), is based on a common architecture for reinforcement learning used in past works including for visual semantic planning [73] and task oriented language grounding [8, 19].”
Learning to Navigate Unseen Environments: Back Translation with Environmental Dropout (2019, background): “…including instruction-based navigation (Chaplot et al., 2018; Anderson et al., 2018b), target-driven navigation (Zhu et al., 2017b; Gupta et al., 2017), embodied question answering (Das et al., 2018), interactive question answering (Gordon et al., 2018), and task planning (Zhu et al., 2017a).”
Motion-Appearance Co-memory Networks for Video Question Answering (2018, neutral): “Many methods have been proposed [37, 5, 14, 21, 36, 7, 3, 1, 33, 23, 4, 28, 17, 13, 40, 38, 41, 42].”
SoundSpaces: Audio-Visual Navigation in 3D Environments (2019, background): “Building on recent embodied visual navigation work [111,41,40,4,64,59], we take a deep reinforcement learning approach, and we introduce audio to the observation.”
Visual Room Rearrangement (2021, background): “…this domain: navigation towards objects [4, 51, 48, 7] or towards a speciﬁc point [1, 38, 47, 36], scene exploration [9, 8], embodied question answering [18, 13], task completion [55], instruction following [2, 41], object manipulation [16, 52], multi-agent coordination [24, 23], and many others.”
Multi-Target Embodied Question Answering (2019, background): “In the vision, language, and AI communities, we are witnessing a shift in focus from internet vision to embodied AI – with the creation of new tasks and benchmarks [7, 2, 14, 35], instantiated on new simulation platforms [21, 28, 32, 33, 20, 6].”
ManipulaTHOR: A Framework for Visual Object Manipulation (2021, result): “In contrast, recent developments in the Embodied AI domain [42, 29] have demonstrated the benefits of using end-to-end learning-based approaches.”
Online Continual Learning For Interactive Instruction Following Agents (2024, background): “…for robotic agents, encompassing navigation (Savva et al., 2019; Deitke et al., 2020; Anderson et al., 2018; Krantz et al., 2020), object interaction (Zhu et al., 2017; Misra et al., 2017; Weihs et al., 2021; Ehsani et al., 2021), and interactive reasoning (Das et al., 2018; Gordon et al., 2018).”
What Should I Do Now? Marrying Reinforcement Learning and Symbolic Planning (2019, methodology): “Additionally, we include navigation as a subtask of planning which [36] omits.”
Visually-Grounded Planning without Vision: Language Models Infer Detailed Plans from High-level Instructions (2020, methodology): “In contrast to previous approaches to visual semantic planning (e.g. Zhu et al., 2017; Fried et al., 2018; Fang et al., 2019), we explore the performance limits of this task solely using goals expressed in natural language as input – that is, without visual input from the virtual environment.”
Occupancy Anticipation for Efficient Exploration and Navigation (2020, background): “More recent work uses deep learning to learn navigation [79,20,56,41,77,75,59,64] or exploration [48,6,57,28,51] policies in an end-to-end fashion.”
Shaping embodied agent behavior with activity-context priors from egocentric video (2021, methodology): “This requires a large number of trials for the agent to chance upon suitable conﬁgurations, and leads to poor sample efﬁciency or learning failures [68, 58].”