Distinct Functional Contributions of Primary Sensory and Association Areas to Audiovisual Integration in Object Categorization

By S. Werner, Uta Noppeney. 2010

202 citations recorded by the source corpus.

Multisensory interactions have been demonstrated in a distributed neural system encompassing primary sensory and higher-order association areas. However, their distinct functional roles in multisensory integration remain unclear. This functional magnetic resonance imaging study dissociated the functional contributions of three cortical levels to multisensory integration in object categorization. Subjects actively categorized or passively perceived noisy auditory and visual signals emanating from everyday actions with objects. The experiment included two 2 × 2 factorial designs that manipulated either (1) the presence/absence or (2) the informativeness of the sensory inputs. These experimental manipulations revealed three patterns of audiovisual interactions. (1) In primary auditory cortices (PACs), a concurrent visual input increased the stimulus salience by amplifying the auditory response regardless of task-context. Effective connectivity analyses demonstrated that this automatic response amplification is mediated via both direct and indirect [via superior temporal sulcus (STS)] connectivity to visual cortices. (2) In STS and intraparietal sulcus (IPS), audiovisual interactions sustained the integration of higher-order object features and predicted subjects' audiovisual benefits in object categorization. (3) In the left ventrolateral prefrontal cortex (vlPFC), explicit semantic categorization resulted in suppressive audiovisual interactions as an index for multisensory facilitation of semantic retrieval and response selection. In conclusion, multisensory integration emerges at multiple processing stages within the cortical hierarchy. The distinct profiles of audiovisual interactions dissociate audiovisual salience effects in PACs, formation of object representations in STS/IPS and audiovisual facilitation of semantic categorization in vlPFC. Furthermore, in STS/IPS, the profiles of audiovisual interactions were behaviorally relevant and predicted subjects' multisensory benefits in performance accuracy.

Subfields: Cognitive Neuroscience, Experimental and Cognitive Psychology, Sensory Systems

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Experimental and Cognitive Psychology

In Sensory Systems

Citations in the corpus

Audiovisual Synchrony Improves Motion Discrimination via Enhanced Connectivity between Early Visual and Auditory Areas (2010, background): “…bimodal neurons in nonhuman primates (Benevento et al., 1977; Bruce et al., 1981; Hikosaka et al., 1988) and more recently in several human brain imaging studies (Calvert et al., 2000, 2001; Beauchamp et al., 2004a,b; Stevenson et al., 2007; Stevenson and James, 2009; Werner and Noppeney, 2010a,b).”
Prior auditory information shapes visual category-selectivity in ventral occipito-temporal cortex (2010, unknown): “If the auditory and visual senses disagree and opt for different responses, this neural accumulation process is protracted leading to an increased BOLD response (Werner and Noppeney, 2010; Noppeney et al., 2010).”
Electrical neuroimaging of memory discrimination based on single-trial multisensory learning (2012, background): “the parallel between the effects obtained by Werner and Noppeney (2010) and our observations of behaviorally independent responses”
Multisensory context portends object memory. (2014, unknown): “Prior findings have implicated these regions in perceptual benefits of multisensory object recognition [7].”
Multisensory integration: flexible use of general operations (2014, unknown): “There are several accounts for how these multiple architectures may be functionally complementary (Driver and Noesselt, 2008; Ghazanfar and Schroeder, 2006; van Atteveldt et al., 2013; Werner and Noppeney, 2010).”
Flexibility and Stability in Sensory Processing Revealed Using Visual-to-Auditory Sensory Substitution (2014, background): “This left lateralized network overlapped with other known left lateralized networks involved in object detection (Werner and Noppeney 2010; Lee and Noppeney 2011), SSA learning (Amedi et al. 2007), attention (Corbetta and Shulman 2002), and language (Hickok and Poeppel 2000).”
Distinct Computational Principles Govern Multisensory Integration in Primary Sensory and Association Cortices. (2016, unknown): “Accumulating evidence has demonstrated that multisensory integration is not deferred to association cortices [17–22] but starts already at the primary, putatively unisensory level [23–28] via thalamo-cortical mechanisms [27], direct connectiv-”
Eccentricity dependent auditory enhancement of visual stimulus detection but not discrimination (2013, background): “…2005; Ghazanfar and Schroeder, 2006), as suggested not only by psychophysical studies (Jaekl and Harris, 2009; Jaekl and Soto-Faraco, 2010) but also by functional imaging work (Giard and Peronnet, 1999; Molholm et al., 2002; Watkins et al., 2006; Noesselt et al., 2010; Werner and Noppeney, 2010).”
The Curious Incident of Attention in Multisensory Integration: Bottom-up vs. Top-down (2016, unknown): “These results suggest that low level temporal properties of the stimuli may determine MSI in a more automatic fashion, while higher order representational integration (e.g. motion, shape) may be more sensitive to topdown effects (Lee and Noppeney, 2014; Lewis and Noppeney, 2010).”
The COGs (context, object, and goals) in multisensory processing (2016, background): “However, naturalistic multisensory stimuli frequently activate the STC together with other areas, such as the planum temporale (speech/script, van Atteveldt et al. 2004) or the inferior parietal sulcus (IPS; tools/animals, Werner and Noppeney 2010a).”