Psychlab: A Psychology Laboratory for Deep Reinforcement Learning Agents

By Joel Z. Leibo, Cyprien de Masson d'Autume, Daniel Zoran, David Amos, Charlie Beattie, Keith Anderson, Antonio García Castañeda, Manuel Sanchez, Simon Green, Audrunas Gruslys and 3 more. 2018

82 citations recorded by the source corpus.

Psychlab is a simulated psychology laboratory inside the first-person 3D game world of DeepMind Lab (Beattie et al. 2016). Psychlab enables implementations of classical laboratory psychological experiments so that they work with both human and artificial agents. Psychlab has a simple and flexible API that enables users to easily create their own tasks. As examples, we are releasing Psychlab implementations of several classical experimental paradigms including visual search, change detection, random dot motion discrimination, and multiple object tracking. We also contribute a study of the visual psychophysics of a specific state-of-the-art deep reinforcement learning agent: UNREAL (Jaderberg et al. 2016). This study leads to the surprising conclusion that UNREAL learns more quickly about larger target stimuli than it does about smaller stimuli. In turn, this insight motivates a specific improvement in the form of a simple model of foveal vision that turns out to significantly boost UNREAL's performance, both on Psychlab tasks, and on standard DeepMind Lab tasks. By open-sourcing Psychlab we hope to facilitate a range of future such studies that simultaneously advance deep reinforcement learning and improve its links with cognitive science.

Subfields: Cognitive Neuroscience, Experimental and Cognitive Psychology, Reinforcement Learning, Cognitive Science & Psychology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Experimental and Cognitive Psychology

In Reinforcement Learning

In Cognitive Science & Psychology

Citations in the corpus

IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures (2018, background): “, 2017), navigation tasks, cognitive (Leibo et al., 2018) and first-person tagging tasks featuring scripted bots as opponents.”
Optimizing agent behavior over long time scales by transporting value (2018, methodology): “In Psychlab Arbitrary Visuomotor Mapping, a task in the Psychlab framework [3], the agent is shown images from a visual memory capacity experiment dataset [4] but in an experimental protocol known as arbitrary visuomotor mapping.”
Unity: A General Platform for Intelligent Agents (2018, methodology): “Furthermore, the engine was also designed to enable ﬁrst-person shooter games and so the environments built using Lab are limited to agents with a ﬁrst-person perspective.”
AGI Safety Literature Review (2018, unknown): “The tests led them to a simple improvement of the UNREAL agent (Leibo et al., 2018).”
Towards Safe Artificial General Intelligence (2018, unknown): “The tests led them to a simple improvement of the UNREAL agent (Leibo et al., 2018).”
Increasing generality in machine learning through procedural content generation (2019, background): “In the Psych-lab environment [87], agents are tested on known tasks from cognitive psychology, such as visual search or object tracking, making the results from simulated agents directly comparable to human results.”
Re-evaluating evaluation (2018, background): “The desire to build increasingly general-purpose agents has led to a proliferation of environments: Mujoco, DM Lab, Open AI Gym, Psychlab and others [12–15].”
Generalization of Reinforcement Learners with Working and Episodic Memory (2019, methodology): “These include cognitive psychology tasks adapted from PsychLab (Leibo et al., 2018) and DMLab (Beattie et al., 2016), and new tasks built with the Unity 3D game engine (Juliani et al., 2018) that require the agent to 1) spot the difference between two scenes; 2) remember the location of a goal and…”
Neuroevolution of self-interpretable agents (2020, background): “[3, 32, 45, 46, 52, 58, 70, 87, 102, 106]) that approach the generalization problem, and future work will continue to develop on these ideas to push the generalization abilities proposed in more general domains [7, 16, 17, 49, 59, 70, 105].”
Autocurricula and the Emergence of Innovation from Social Interaction: A Manifesto for Multi-Agent Intelligence Research (2019, background): “…mazes (Mirowski et al., 2016; Savinov et al., 2018), foraging over rough, naturalistic terrain (Espeholt et al., 2018; Hessel et al., 2018), and tests drawn from the neuroscience and cognitive psychology literatures like visual search (Leibo et al., 2018) and memory recall (Wayne et al., 2018).”
Using Unity to Help Solve Intelligence (2020, background): “For example, DeepMind Lab proved an excellent platform for exploring complex navigational tasks (Banino et al., 2018) and classical laboratory psychological experiments (Leibo et al., 2018), among others, but is limited to ﬁrst-person perspectives and primitive physics.”
Arena: A General Evaluation Platform and Building Toolkit for Multi-Agent Intelligence (2019, background): “DeepMind Lab [6], Psychlab [51], and Malmo [39] are the most possible choices when making a customizable general evaluation platform.”
Kickstarting Deep Reinforcement Learning (2018, background): “…long-term memory of maze structure, and in some cases planning and prediction), tasks of foraging for food items in naturalistic environments (challenging the visual system of an agent), and tasks based on classical experiments from cognitive neuroscience like visual search (Leibo et al., 2018).”
Prefrontal cortex as a meta-reinforcement learning system (2018, background): “(2018).(74) An 84x84 pixel input represented a simulated computer screen (see Figure 6A and Supplementary Video 1; https://youtu.”