Mapping Sub-Second Structure in Mouse Behavior.

By Alexander B. Wiltschko, Matthew J. Johnson, Giuliano Iurilli, Ralph E. Peterson, Jesse M. Katon, Stan Pashkovski, V. Abraira, Ryan P. Adams, Sandeep R. Datta. 2015

735 citations recorded by the source corpus.

Summary Complex animal behaviors are likely built from simpler modules, but their systematic identification in mammals remains a significant challenge. Here we use depth imaging to show that three-dimensional (3D) mouse pose dynamics are structured at the sub-second timescale. Computational modeling of these fast dynamics effectively describes mouse behavior as a series of reused and stereotyped modules with defined transition probabilities. We demonstrate this combined 3D imaging and machine learning method can be used to unmask potential strategies employed by the brain to adapt to the environment, to capture both predicted and previously-hidden phenotypes caused by genetic or neural manipulations, and to systematically expose the global structure of behavior within an experiment. This work reveals that mouse body language is built from identifiable components and is organized in a predictable fashion; deciphering this language establishes an objective framework for characterizing the influence of environmental cues, genes and neural activity on behavior.

Subfields: Cell Biology, Cellular and Molecular Neuroscience, Cognitive Neuroscience

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cell Biology

In Cellular and Molecular Neuroscience

In Cognitive Neuroscience

Citations in the corpus

Composing graphical models with neural networks for structured representations and fast inference (2016, methodology): “A natural representation from ethology [3] is that the mouse’s behavior is divided into brief, reused actions, such as darts, rears, and grooming bouts.”
Human-level performance in 3D multiplayer games with population-based reinforcement learning (2018, methodology): “We developed an unsupervised method to automatically discover and quantitatively characterise temporally extended behaviour patterns, inspired by models of mouse behaviour (73), which groups short game-play sequences into behavioural clusters (Figure S9, Supplementary Video https://youtu.”
Computational neuroethology: a call to action (2019, methodology): “…is modeled as a continuous auto-regressive process in pose space, while the components themselves (whose duration distributions are flexibly modeled based upon the data) are modeled using an HMM (Johnson et al., 2016; Markowitz et al., 2018; Pisanello et al., 2017; Wiltschko et al., 2015).”
Finding, visualizing, and quantifying latent structure across diverse animal vocal repertoires (2020, background): “Similarly, latent models are playing an increasingly important role in computational ethology (Brown & De Bivort, 2018), where characterizations of animal movements and behaviors have uncovered complex sequential organization (Marques et al., 2018; Berman et al., 2016; Wiltschko et al., 2015).”
Using DeepLabCut for 3D markerless pose estimation across species and behaviors (2018, methodology): “The outputs from DeepLabCut can also interfaced with behavioral clustering tools such as JAABA [43], MotionMapper [44], MoSeq [45], or other clustering approaches such as iterative denoising tree (IDT) methods [46, 47], and more [48].”
Structured VAEs: Composing Probabilistic Graphical Models and Variational Autoencoders (2016, methodology): “Learning interpretable representations for such data, and studying how those representations change as the animal’s genetics are edited or its brain chemistry altered, can create powerful behavioral phenotyping tools for neuroscience and for high-throughput drug discovery (Wiltschko et al., 2015).”