Deep neuroethology of a virtual rodent

By Josh Merel, Diego E. Aldarondo, Jesse D. Marshall, Yuval Tassa, Greg Wayne, Bence Olveczky. 2019

80 citations recorded by the source corpus.

Parallel developments in neuroscience and deep learning have led to mutually productive exchanges, pushing our understanding of real and artificial neural networks in sensory and cognitive systems. However, this interaction between fields is less developed in the study of motor control. In this work, we develop a virtual rodent as a platform for the grounded study of motor activity in artificial models of embodied control. We then use this platform to study motor activity across contexts by training a model to solve four complex tasks. Using methods familiar to neuroscientists, we describe the behavioral representations and algorithms employed by different layers of the network using a neuroethological approach to characterize motor activity relative to the rodent's behavior and goals. We find that the model uses two classes of representations which respectively encode the task-specific behavioral strategies and task-invariant behavioral kinematics. These representations are reflected in the sequential activity and population dynamics of neural subpopulations. Overall, the virtual rodent facilitates grounded collaborations between deep reinforcement learning and motor neuroscience.

Subfields: Cognitive Neuroscience, Social Psychology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Social Psychology

Citations in the corpus

Critic Regularized Regression (2020, background): “[23] Josh Merel, Diego Aldarondo, Jesse Marshall, Yuval Tassa, Greg Wayne, and Bence Ölveczky.”
RL Unplugged: Benchmarks for Offline Reinforcement Learning (2020, methodology): “For the rodent task, we use the same training scheme as in Merel et al. (2020).”
dm_control: Software and Tasks for Continuous Control (2020, methodology): “See (Merel et al., 2020) for initial research training a policy to control this model using visual inputs and analysing the resulting neural representations.”
RL Unplugged: A Suite of Benchmarks for Ofﬂine Reinforcement Learning (2020, methodology): “These tasks are made up of the corridor locomotion tasks involving the CMU Humanoid [Tassa et al., 2020], for which prior efforts have either used motion capture data [Merel et al., 2019a,b] or training from scratch [Song et al., 2020].”
Catalyzing next-generation Artificial Intelligence through NeuroAI (2022, unknown): “Additionally, detailed bio-mechanical measurements support highly realistic animal body models, complete with skeletal constraints, muscles, tendons, and paw features 58 .”
Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depth (2020, background): “…et al., 2019) to revealing hidden linguistic structure in large language models (Bau et al., 2019; Kudugunta et al., 2019; Wu et al., 2019; 2020) to applications in neuroscience (Shi et al., 2019; Li et al., 2019; Merel et al., 2019; Zhang & Bellec, 2020) and medicine (Raghu et al., 2019).”