The co-occurrence of multisensory facilitation and cross-modal conflict in the human brain.

By A. Diaconescu, Claude Alain, Anthony Randal McIntosh. 2011

77 citations recorded by the source corpus.

Perceptual objects often comprise a visual and auditory signature that arrives simultaneously through distinct sensory channels, and cross-modal features are linked by virtue of being attributed to a specific object. Continued exposure to cross-modal events sets up expectations about what a given object most likely "sounds" like, and vice versa, thereby facilitating object detection and recognition. The binding of familiar auditory and visual signatures is referred to as semantic, multisensory integration. Whereas integration of semantically related cross-modal features is behaviorally advantageous, situations of sensory dominance of one modality at the expense of another impair performance. In the present study, magnetoencephalography recordings of semantically related cross-modal and unimodal stimuli captured the spatiotemporal patterns underlying multisensory processing at multiple stages. At early stages, 100 ms after stimulus onset, posterior parietal brain regions responded preferentially to cross-modal stimuli irrespective of task instructions or the degree of semantic relatedness between the auditory and visual components. As participants were required to classify cross-modal stimuli into semantic categories, activity in superior temporal and posterior cingulate cortices increased between 200 and 400 ms. As task instructions changed to incorporate cross-modal conflict, a process whereby auditory and visual components of cross-modal stimuli were compared to estimate their degree of congruence, multisensory processes were captured in parahippocampal, dorsomedial, and orbitofrontal cortices 100 and 400 ms after stimulus onset. Our results suggest that multisensory facilitation is associated with posterior parietal activity as early as 100 ms after stimulus onset. However, as participants are required to evaluate cross-modal stimuli based on their semantic category or their degree of congruence, multisensory processes extend in cingulate, temporal, and prefrontal cortices.

Subfields: Cognitive Neuroscience, Experimental and Cognitive Psychology, Sensory Systems

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Experimental and Cognitive Psychology

In Sensory Systems

Citations in the corpus

Expectation Learning for Adaptive Crossmodal Stimuli Association (2018, background): “tion allows us to enhance our perceptual experience [1] also for unisensory stimuli [2], to solve associative incongruence and conflicts [3], and to learn new concepts [4].”
The effect of synesthetic associations between the visual and auditory modalities on the Colavita effect (2015, background): “…for semantic incongruent stimuli are slower for incongruent audiovisual stimuli than for congruent stimuli when participants evaluated the semantic congruency between auditory and visual stimuli, whereas in a stimulus detection task, semantic congruency had no effect on RT (Diaconescu et al. 2011).”
The COGs (context, object, and goals) in multisensory processing (2016, background): “Most evidence suggests that the brain is sensitive to (in)congruence of object semantic features in multisensory stimuli starting at approximately 150–200 ms post-stimulus (evoked responses: Raij et al. 2000; Molholm et al. 2004; Diaconescu et al. 2011; but see Naci et al. 2012 for effects <100 ms; ”
Contextual factors multiplex to control multisensory processes (2016, background): “These areas have been repeatedly observed in studies of audiovisual integration [e.g., Calvert et al., 2000; Cappe et al., 2010, 2012; Dhamala et al., 2007), with the orbitofrontal cortices implicated in multisensory conflict [Diaconescu et al., 2011].”
Does crossmodal correspondence modulate the facilitatory effect of auditory cues on visual search? (2012, methodology): “Some support for a flexible “multistage” model of cross-modal integration comes from a recent magnetic encephalography study by Diaconescu, Alain, and McIntosh (2011).”