Neural Modular Control for Embodied Question Answering

By Abhishek Das, Georgia Gkioxari, Stefan Lee, Devi Parikh, Dhruv Batra. 2018

147 citations recorded by the source corpus.

We present a modular approach for learning policies for navigation over long planning horizons from language input. Our hierarchical policy operates at multiple timescales, where the higher-level master policy proposes subgoals to be executed by specialized sub-policies. Our choice of subgoals is compositional and semantic, i.e. they can be sequentially combined in arbitrary orderings, and assume human-interpretable descriptions (e.g. 'exit room', 'find kitchen', 'find refrigerator', etc.). We use imitation learning to warm-start policies at each level of the hierarchy, dramatically increasing sample efficiency, followed by reinforcement learning. Independent reinforcement learning at each level of hierarchy enables sub-policies to adapt to consequences of their actions and recover from errors. Subsequent joint hierarchical training enables the master policy to adapt to the sub-policies. On the challenging EQA (Das et al., 2018) benchmark in House3D (Wu et al., 2018), requiring navigating diverse realistic indoor environments, our approach outperforms prior work by a significant margin, both in terms of navigation and question answering.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

In Reinforcement Learning

Citations in the corpus

ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks (2019, background): “These challenges may be approachable by models that exploit hierarchy [8, 26], modularity [4, 16], and structured reasoning and planning [6].”
A Survey of Embodied AI: From Simulators to Research Tasks (2021, methodology): “[104] and [105] further improved the PACMAN model by the Neural Modular Control (NMC) where the higher-level master policy proposes seman-”
A Survey of Reinforcement Learning Informed by Natural Language (2019, methodology): “Embodied Q&A (EQA) extends this setting, by requiring agent to explore and navigate the environment in order to answer queries [Das et al. , 2018a; Gordon et al. , 2018 ] , for example, “How many mugs are in the kitchen?” or “Is there a tomato in the fridge?”.”
Language as an Abstraction for Hierarchical Deep Reinforcement Learning (2019, methodology): “While prior work has made use of language-based sub-goal policies in hierarchical RL [51, 13], the instruction representation used lack the needed diversity to beneﬁt from the compositionality of language over one-hot goal representations.”
ELLA: Exploration through Learned Language Abstraction (2021, methodology): “This makes it difficult to use explicitly modular approaches (Arumugam et al., 2017; Andreas et al., 2017; Das et al., 2018) out of the box; instead need to be able to explore using a mixture of these low-level instructions as well as primitive actions.”
Aligning Cyber Space With Physical World: A Comprehensive Survey on Embodied AI (2024, unknown): “Improving data efﬁciency, enhancing transfer learning techniques, and incorporating real-world priors can also enable more accurate predictions and adaptive decision-making.”
Embodied Question Answering in Photorealistic Environments With Point Cloud Perception (2019, background): “To this end, a number of recent works have proposed goal-driven, perception-based tasks situated in simulated environments to develop such agents [1–10].”
Vision-Based Navigation With Language-Based Assistance via Imitation Learning With Indirect Intervention (2018, background): “[19] propose a hierarchical solution for this task where each level of the hierarchy is independently warmed up with imitation learning and further improved with reinforcement learning.”
Binding Language Models in Symbolic Languages (2022, background): “Andreas et al. (2016); Hu et al. (2017); Das et al. (2018) and Gupta et al. (2019) generate programs that are further softly executed by the corresponding neural modules.”
Auxiliary Tasks Speed Up Learning PointGoal Navigation (2020, unknown): “Su-pervised auxiliary tasks expose privileged information to the agent (such as depth [22, 27, 28, 29]).”
From Seeing to Moving: A Survey on Learning for Visual Indoor Navigation (VIN) (2020, methodology): “More advanced variations can be found in [Das et al., 2018b; Yu et al., 2019; Wijmans et al., 2019].”
An Exploration of Embodied Visual Exploration (2020, unknown): “In order to train the policy architecture, we adapt current methods [18,19,16] and divide the learning into two stages: (1) imitation learning, and (2) fine-tuning with reinforcement learning.”
Embodied Vision-and-Language Navigation with Dynamic Convolutional Filters (2019, background): “[3], with the aim of connecting the research efforts on vision-and-language understanding [2, 4, 6, 7, 13, 23, 26] with the raising area of embodied AI [1, 8, 9, 25].”
Bayesian Relational Memory for Semantic Visual Navigation (2019, background): “There are also works studying visual navigation under natural language guidance, including instruction following [6, 39, 23, 55, 3] and question answering [11, 10, 1].”
A Cordial Sync: Going Beyond Marginal Policies for Multi-Agent Embodied Tasks (2020, background): “Embodied navigation also aids AI agents to develop behavior such as instruction following [38,4,82,95,3], city navigation [18,64,63,94], question answering [21,22,34,97,24], and active visual recognition [105,104].”