The NetHack Learning Environment

By Heinrich Kuttler, Nantas Nardelli, Alexander H. Miller, Roberta Raileanu, Marco Selvatici, Edward Grefenstette, Tim Rocktaschel. 2020

233 citations recorded by the source corpus.

Progress in Reinforcement Learning (RL) algorithms goes hand-in-hand with the development of challenging environments that test the limits of current methods. While existing RL environments are either sufficiently complex or based on fast simulation, they are rarely both. Here, we present the NetHack Learning Environment (NLE), a scalable, procedurally generated, stochastic, rich, and challenging environment for RL research based on the popular single-player terminal-based roguelike game, NetHack. We argue that NetHack is sufficiently complex to drive long-term research on problems such as exploration, planning, skill acquisition, and language-conditioned RL, while dramatically reducing the computational resources required to gather a large amount of experience. We compare NLE and its task suite to existing alternatives, and discuss why it is an ideal medium for testing the robustness and systematic generalization of RL agents. We demonstrate empirical success for early stages of the game using a distributed Deep RL baseline and Random Network Distillation exploration, alongside qualitative analysis of various agents trained in the environment. NLE is open source at this https URL.

Subfields: Artificial Intelligence, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Reinforcement Learning

Citations in the corpus

ALFWorld: Aligning Text and Embodied Environments for Interactive Learning (2020, methodology): “Embodied Language Learning : To address language learning in embodied domains, a number of interactive environments have been proposed: BabyAI (Chevalier-Boisvert et al., 2019), Room2Room (Anderson et al., 2018b), ALFRED (Shridhar et al., 2020), InteractiveQA (Gordon et al., 2018), EmbodiedQA (Das e”
WebArena: A Realistic Web Environment for Building Autonomous Agents (2023, background): “(Kolve et al., 2017; Shridhar et al., 2020; Puig et al., 2018) and extends to gaming environments (Fan et al., 2022; Küttler et al., 2020), where the environment mechanisms often diverge from human objectives.”
Situated Dialogue Learning through Procedural Environment Generation (2021, background): “We’d also like to note that most works using procedural generation to benchmark RL agents such as Cobbe et al. (2020); Küttler et al. (2020); Samvelyan et al. (2021) rely on the underlying richness of the game engines to generate novel environments as opposed to learning a procedural generation…”
RLDS: an Ecosystem to Generate, Share and Use Datasets in Reinforcement Learning (2021, background): “Some works include datasets for discrete action-environments like games [19, 28].”
Improving Intrinsic Exploration with Language Abstractions (2022, background): “Many modern RL environments are pre-equipped with language, including NetHack/MiniHack [28, 41], text-based games [15, 45, 52], and in fact most video games in general.”
BeBold: Exploration Beyond the Boundary of Explored Regions (2020, methodology): “We evaluate BeBold on challenging procedurally-generated environment MiniGrid (Chevalier-Boisvert et al., 2018) and the hard-exploration environment NetHack (K¨uttler et al., 2020).”
CORA: Benchmarks, Baselines, and Metrics as a Platform for Continual Reinforcement Learning Agents (2021, methodology): “Procgen (Cobbe et al., 2020) and MiniHack (Küttler et al., 2020; Samvelyan et al., 2021), two of our other benchmarks, are procedurally-generated, like Jelly Bean World (Platanios et al., 2020) which is a procedurally-generated 2D gridworld proposed as a testbed for continual learning agents.”
Human-Timescale Adaptation in an Open-Ended Task Space (2023, background): “Indeed, in the past few years a series of challenging PCG environments have been proposed (Juliani et al., 2019; Cobbe et al., 2020; K¨uttler et al., 2020; Samvelyan et al., 2021; Chevalier-Boisvert et al., 2018; Hafner, 2022; Deitke et al., 2022), mostly focusing on testing and improving…”
MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge (2022, methodology): “Prior works include maze-style worlds [121, 129, 61], purely text-based game [69], grid worlds [21, 16], browser/GUI-based environments [108, 124], and indoor simulators for robotics [1, 107, 114, 34, 110, 99, 89].”
AgentBench: Evaluating LLMs as Agents (2023, background): “More recently, attempts on embodied agents [63; 35; 2] have employed complicated multi-modal simulators based on games [41; 22], GUI [69; 81], and indoor scenes [68; 74].”
Automatic Data Augmentation for Generalization in Reinforcement Learning (2021, background): “A recent body of work has pointed out the problem of overﬁtting in deep RL [53, 46, 34, 51, 71, 75, 50, 11, 12, 33, 52, 40, 23].”