BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learning

By Maxime Chevalier-Boisvert, Dzmitry Bahdanau, Salem Lahlou, Lucas Willems, Chitwan Saharia, Thien Huu Nguyen, Yoshua Bengio. 2018

335 citations recorded by the source corpus.

Allowing humans to interactively train artificial agents to understand language instructions is desirable for both practical and scientific reasons, but given the poor data efficiency of the current learning methods, this goal may require substantial research efforts. Here, we introduce the BabyAI research platform to support investigations towards including humans in the loop for grounded language learning. The BabyAI platform comprises an extensible suite of 19 levels of increasing difficulty. The levels gradually lead the agent towards acquiring a combinatorially rich synthetic language which is a proper subset of English. The platform also provides a heuristic expert agent for the purpose of simulating a human teacher. We report baseline results and estimate the amount of human involvement that would be required to train a neural network-based agent on some of the BabyAI levels. We put forward strong evidence that current deep learning methods are not yet sufficiently sample efficient when it comes to learning a language with compositional properties.

Subfields: Artificial Intelligence, Natural Language Processing, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Reinforcement Learning

Citations in the corpus

LILA: Language-Informed Latent Actions (2021, background): “Paramount is sample efficiency; imitation learning approaches often require hundreds to thousands of demonstrations for learning to navigate [35, 44], and reinforcement learning approaches can require millions of episodes of experience to learn robust policies [12, 45].”
CausalWorld: A Robotic Manipulation Benchmark for Causal Structure and Transfer Learning (2020, methodology): “…were used in experiments in the generalization for RL literature, which have played an important role in advancing the ﬁeld (Packer et al., 2018; Rajeswaran et al., 2017; Pinto et al., 2017; Yu et al., 2017; Henderson et al., 2017a; Dulac-Arnold et al., 2020; Chevalier-Boisvert et al., 2018).”
Object Files and Schemata: Factorizing Declarative and Procedural Knowledge in Dynamical Systems (2020, methodology): “We use the GotoObjMaze environment (i.e MiniGrid-GotoObjMaze-v0) from [Chevalier-Boisvert et al., 2018].”
Recurrent Independent Mechanisms (2019, background): “These tasks are difficult to solve (Chevalier-Boisvert et al., 2018) with standard RL algorithms, due to (1) the partial observability of the environment and (2) the sparsity of the reward, given that the agent receives a reward only after reaching the goal.”
Learning First-Order Symbolic Representations for Planning from the Structure of the State Space (2019, background): “Deep reinforcement learning (DRL) methods [30], on the other hand, generate policies over high-dimensional perceptual spaces like images, without using any prior symbolic knowledge [19, 10, 15].”
Continual Reinforcement Learning in 3D Non-stationary Environments (2019, neutral): “MiniGrid [7] 6×6×3 no “Competencies” introduced in a curriculum.”
Human Instruction-Following with Deep Reinforcement Learning via Transfer-Learning from Text (2020, methodology): “Much recent work applies deep learning and policy optimization in end-to-end approaches to learning instruction following [Chaplot et al., 2018, Oh et al., 2017, Bahdanau et al., 2018, Chevalier-Boisvert et al., 2018, Yu et al., 2018b,a, Jiang et al., 2019].”
A Generalist Agent (2022, background): “On BabyAI (Chevalier-Boisvert et al., 2018) Gato achieves over 80% of expert score for nearly all levels2.”
Help, Anna! Visual Navigation with Natural Multimodal Assistance via Retrospective Curiosity-Encouraging Imitation Learning (2019, background): “, 2019), or mimic human verbal assistance with primitive, highly scripted language (Nguyen et al., 2019; Chevalier-Boisvert et al., 2019).”
Inductive biases for deep learning of higher-level cognition (2020, background): “Studying this kind of setup was the motivation for our work on the Baby AI environment (Chevalier-Boisvert et al., 2018).”
Language Conditioned Imitation Learning Over Unstructured Data (2020, background): “binary pick and place primitives [18, 48]), or synthetic predefined language [17, 6, 20, 54].”
Can Small and Synthetic Benchmarks Drive Modeling Innovation? A Retrospective Study of Question Answering Modeling Approaches (2021, neutral): “Synthetic benchmarks only use synthetic data during training and testing for the purpose of comparing modeling approaches (Weston et al., 2016; Lake and Baroni, 2018; Chevalier-Boisvert et al., 2018; Saxton et al., 2019; Kim and Linzen, 2020; Ruis et al., 2020; Hui et al., 2020; Keysers et al., 2020”
The NetHack Learning Environment (2020, background): “Recent advances in (Deep) Reinforcement Learning (RL) have been driven by the development of novel simulation environments, such as the Arcade Learning Environment (ALE) [9], StarCraft [64, 69], BabyAI [16], Obstacle Tower [38], Minecraft [37, 29, 35], and Procgen Benchmark [18].”
Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Tasks (2023, methodology): “Furthermore, research has built upon Minigrid for new environments, e.g. BabyAI [4] where the environment is constructed and evaluated using natural language-based instructions.”
Interactive Learning from Activity Description (2021, background): “2 But the trade-off of using such limited feedback is that the task-learning effort of the agent increases; state-of-the-art RL algorithms are notorious for their high sample complexity (Hermann et al., 2017; Chaplot et al., 2018; Chevalier-Boisvert et al., 2019).”
MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge (2022, methodology): “[17] Maxime Chevalier-Boisvert, Dzmitry Bahdanau, Salem Lahlou, Lucas Willems, Chitwan Saharia, Thien Huu Nguyen, and Yoshua Bengio.”
A Benchmark for Systematic Generalization in Grounded Language Understanding (2020, methodology): “…of compositional generalization for robustness and sample ef-ﬁciency, and responded by building synthetic environments to evaluate aspects of this skill (Johnson et al., 2017b; Lake & Baroni, 2018; Loula et al., 2018; Bah-danau et al., 2018; Hill et al., 2019; Chevalier-Boisvert et al., 2019).”