Audiovisual Synchrony Improves Motion Discrimination via Enhanced Connectivity between Early Visual and Auditory Areas

By R. Lewis, Uta Noppeney. 2010

170 citations recorded by the source corpus.

Audiovisual synchrony enables integration of dynamic visual and auditory signals into a more robust and reliable multisensory percept. In this fMRI study, we investigated the neural mechanisms by which audiovisual synchrony facilitates shape and motion discrimination under degraded visual conditions. Subjects were presented with visual patterns that were rotated by discrete increments at irregular and unpredictable intervals while partially obscured by a dynamic noise mask. On synchronous trials, each rotation coincided with an auditory click. On asynchronous trials, clicks were noncoincident with the rotational movements (but with identical temporal statistics). Subjects discriminated shape or rotational motion profile of the partially hidden visual stimuli. Regardless of task context, synchronous signals increased activations bilaterally in (1) calcarine sulcus (CaS) extending into ventral occipitotemporal cortex and (2) Heschl's gyrus extending into planum temporale (HG/PT) compared with asynchronous signals. Adjacent to these automatic synchrony effects, synchrony-induced activations in lateral occipital (LO) regions were amplified bilaterally during shape discrimination and in the right posterior superior temporal sulcus (pSTS) during motion discrimination. Subjects' synchrony-induced benefits in motion discrimination significantly predicted blood oxygenation level-dependent synchrony effects in V5/hMT+. According to dynamic causal modeling, audiovisual synchrony increased connectivity between CaS and HG/PT bidirectionally, whereas shape and motion tasks increased forwards connectivity from CaS to LO or to pSTS, respectively. To increase the salience of partially obscured moving objects, audiovisual synchrony may amplify visual activations by increasing the connectivity between low level visual and auditory areas. These automatic synchrony-induced response amplifications may then be gated to higher order areas according to behavioral relevance and task context.

Subfields: Cognitive Neuroscience, Experimental and Cognitive Psychology, Sensory Systems

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Experimental and Cognitive Psychology

In Sensory Systems

Citations in the corpus

Early Experience & Multisensory Perceptual Narrowing (2014, background): “For example, studies have found that synchronous sounds can enhance the detection of partly occluded objects through the initial amplification of responses in primary as well as higher-order visual and auditory cortices and that they can do so regardless of task-context (Lewis & Noppeney, 2010).”
Distinct Computational Principles Govern Multisensory Integration in Primary Sensory and Association Cortices. (2016, unknown): “Accumulating evidence has demonstrated that multisensory integration is not deferred to association cortices [17–22] but starts already at the primary, putatively unisensory level [23–28] via thalamo-cortical mechanisms [27], direct connectiv-”
Sensory reliability shapes perceptual inference via two mechanisms. (2015, unknown): “…that sensory signals should only be integrated when they are close in time, space, and structure (Lee & Noppeney, 2014; Lewald & Guski, 2003; Lewis & Noppeney, 2010; Noppeney, Ostwald, & Werner, 2010; Roach, Heron, & McGraw, 2006; Rohe & Noppeney, 2015; Slutsky & Recanzone, 2001; Wallace et…”
Combined diffusion-weighted and functional magnetic resonance imaging reveals a temporal-occipital network involved in auditory-visual object processing (2013, background): “Functional connectivity MRI showed that BOLD signals between early sensory cortices are highly correlated whereas limited correlation was observed in other brain regions (Eckert et al., 2008; Lewis and Noppeney, 2010; Werner and Noppeney, 2010a).”
A Psychophysical Investigation of Differences between Synchrony and Temporal Order Judgments (2013, unknown): “, [34,44–47]).”
Oscillatory synchronization in large-scale cortical networks predicts perception. (2011, unknown): “These results accord well with recent accounts of cross-modal processing that emphasize the role of recurrent interactions between processing streams traditionally considered as unimodal as well as between early sensory and higher-order multimodal processing stages (Driver and Noesselt, 2008; Driver”
The multisensory function of the human primary visual cortex. (2016, background): “…reported increased coupling between auditory and primary visual cortices, particularly under conditions of synchronous stimulation across the senses (Lewis and Noppeney, 2010; Tyll et al., 2013), that may perhaps be mediated by thalamic circuits (Noesselt et al., 2010; Bonath et al., 2013).”
How prior expectations shape multisensory perception (2016, background): “…integration (Bonath et al., 2014; Cappe et al., 2012; Hein et al., 2007; Körding et al., 2007; Laurienti et al., 2004; Lee and Noppeney, 2014, 2011; Lewis and Noppeney, 2010; Magnotti et al., 2013; Munhall et al., 1996; Stevenson et al., 2012; Wassenhove et al., 2007; Zilber et al., 2014).”
Music expertise shapes audiovisual temporal integration windows for speech, sinewave speech, and music (2014, background): “…sensory signals do not have to be precisely synchronous, they have to co-occur within a certain temporal integration window in order to be integrated into a unified percept (Stein et al., 1993; Spence and Squire, 2003; Noesselt et al., 2007, 2008; Lewis and Noppeney, 2010; Stevenson et al., 2011).”