Causal Inference in Multisensory Perception

By Konrad Paul Kording, Ulrik R Beierholm, Wei Ji Ma, Steven R. Quartz, Josh Tenenbaum, Ladan Shams. 2007

1,070 citations recorded by the source corpus.

Perceptual events derive their significance to an animal from their meaning about the world, that is from the information they carry about their causes. The brain should thus be able to efficiently infer the causes underlying our sensory events. Here we use multisensory cue combination to study causal inference in perception. We formulate an ideal-observer model that infers whether two sensory cues originate from the same location and that also estimates their location(s). This model accurately predicts the nonlinear integration of cues by human subjects in two auditory-visual localization tasks. The results show that indeed humans can efficiently infer the causal structure as well as the location of causes. By combining insights from the study of causal inference with the ideal-observer approach to sensory cue combination, we show that the capacity to infer causal structure is not limited to conscious, high-level cognition; it is also performed continually and effortlessly in perception.

Subfields: Cognitive Neuroscience, Experimental and Cognitive Psychology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Experimental and Cognitive Psychology

Citations in the corpus

A Neurorobotic Experiment for Crossmodal Conflict Resolution in Complex Environments * (2018, background): “, going beyond the typically used simplified AV stimuli such as lights and clicks [9].”
Assessing the Role of the ‘Unity Assumption’ on Multisensory Integration: A Review (2017, methodology): “Frontiers in Psychology | www.frontiersin.org 8 March 2017 | Volume 8 | Article 445 light (that is, a counter-ventriloquism effect was obtained; see also Körding et al., 2007; Rohe and Noppeney, 2015b).”
Distinct Computational Principles Govern Multisensory Integration in Primary Sensory and Association Cortices. (2016, neutral): “Current Biology 26, 509 disparity) [4, 10, 11].”
The Brain’s Tendency to Bind Audiovisual Signals Is Stable but Not General (2016, neutral): “On a given day, each subject participated in two tasks: a temporal-numerosity judgment task (Shams et al., 2000, 2005) and a spatial localization task (Körding et al., 2007; Wozny et al., 2010).”
If I Were You: Perceptual Illusion of Body Swapping (2008, positive): “According to recent statistical models of perception, the interpretation of multisensory signals could be sped up and made more precise by incorporating memory information (‘priors’) in the decision process [44,45,46].”
Computational mechanisms of sensorimotor control. (2011, neutral): “Such a model of multisensory integration is supported by experimental studies of size estimation from visual and haptic cues (Ernst and Banks, 2002), location from visual and auditory cues (Kö rding et al., 2007b), and has been suggested to explain ventriloquism (Alais and Burr, 2004).”
Look, Listen, and Act: Towards Audio-Visual Embodied Navigation (2019, background): “that humans, even in the young-child ages, are remarkably capable of capturing the correspondences between different signal modalities to arrive at a thorough understanding of natural phenomena [1], [2].”
Sensory reliability shapes perceptual inference via two mechanisms. (2015, methodology): “We employed a CI model of audiovisual perception (Koerding et al., 2007).”
Structure Learning in a Sensorimotor Association Task (2010, neutral): “These previous studies focused on more complicated learning problems in which the higher-level inferences made through hierarchical Bayesian inference concern very abstract forms of knowledge, although there have also been studies that used Bayesian methods to explain causal inference in perception ”
Spatiotemporal Movement Planning and Rapid Adaptation for Manual Interaction (2013, methodology): “6) similar to causal inference [24] in multimodal perception.”
Humans' Multisensory Perception, from Integration to Segregation, Follows Bayesian Inference (2011, methodology): “In the spatial localization experiment (Körding et al., 2007), observers were presented in each trial with either a brief visual stimulus in one of five locations (along the horizontal direction in the frontoparallel plane, ranging from -10° to +10° of visual field equally spaced), or a brief sound…”
How prior expectations shape multisensory perception (2016, methodology): “Based on models of Bayesian Causal Inference (Körding et al., 2007; Magnotti et al., 2013; Shams and Beierholm, 2010), we hypothesized that participants would form stronger congruency expectations (i.e. a higher common source prior) in congruent than incongruent contexts.”