What You See Is Not (Always) What You Hear: Induced Gamma Band Responses Reflect Cross-Modal Interactions in Familiar Object Recognition

By Shlomit Yuval-Greenberg, Leon Y. Deouell. 2007

133 citations recorded by the source corpus.

Gamma-band responses (GBRs) are hypothesized to reflect neuronal synchronous activity related to activation of object representations. However, it is not known whether synchrony in the gamma range is also related to multisensory object processing. We investigated the effect of semantic congruity between auditory and visual information on the human GBR. The paradigm consisted of a simultaneous presentation of pictures and vocalizations of animals, which were either congruent or incongruent. EEG was measured in 17 students while they attended either the auditory or the visual stimulus and performed a recognition task. Behavioral results showed a congruity effect, indicating that information from the unattended modality affected behavior. Irrelevant visual information affected auditory recognition more than irrelevant auditory information affected visual recognition, suggesting a bias toward reliance on visual information in object recognition. Whereas the evoked (phase-locked) GBR was unaffected by congruity, the induced (non-phase-locked) GBR was increased for congruent compared with incongruent stimuli. This effect was independent of the attended modality. The results show that integration of information across modalities, based on semantic congruity, is associated with enhanced synchronized oscillations at the gamma band. This suggests that gamma-band oscillations are related not only to low-level unimodal integration but also to the formation of object representations at conceptual multisensory levels.

Subfields: Cognitive Neuroscience, Experimental and Cognitive Psychology, Sensory Systems

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Experimental and Cognitive Psychology

In Sensory Systems

Citations in the corpus

Assessing the Role of the ‘Unity Assumption’ on Multisensory Integration: A Review (2017, background): “…of enhancement in their participants’ behavioral performance or brain activities will be observed (Laurienti et al., 2004; Molholm et al., 2004; Taylor et al., 2006; Hein et al., 2007; Yuval-Greenberg and Deouell, 2007; Adam and Noppeney, 2010; Chen and Spence, 2010; Werner and Noppeney, 2010).”
The effect of synesthetic associations between the visual and auditory modalities on the Colavita effect (2015, background): “Neuroimaging and electrophysiological studies also found semantic influences on the neural correlates of audiovisual integration (Molholm et al. 2004; Hein et al. 2007; Stekelenburg and Vroomen 2007; Yuval-Greenberg and Deouell 2007).”
Top-down control and early multisensory processes: chicken vs. egg (2015, background): “1 To our knowledge semantic congruence does not modulate eMSI (Fort et al., 2002b; Molholm et al., 2004; Yuval-Greenberg and Deouell, 2007).”
Single-trial multisensory memories affect later auditory and visual object discrimination. (2015, unknown): “Similarly, in terms of RTs, we proceeded to a 2x4 (Task Modality by Encounter Context) repeated measures ANOVA, to test for sensory modality related differences (Yuval-Greenberg & Deouell, 2007, 2009).”
Transient induced gamma-band response in EEG as a manifestation of miniature saccades. (2008, methodology): “…(Gruber and Mü ller, 2005), the category of the object presented (Zion-Golumbic and Bentin, 2007), the cross-modal congruity of object information (Yuval-Greenberg and Deouell, 2007), conscious detection of an object in a display (Rodriguez et al., 1999; Tallon-Baudry et al., 1997), and…”
Prior auditory information shapes visual category-selectivity in ventral occipito-temporal cortex (2010, unknown): “…was selected individually for each stimulus to adjust the available object information and maximize the incongruency effects (n.b. auditory information interferes more strongly with visual object recognition when the visual signal is rendered unreliable, Yuval-Greenberg and Deouell, 2007, 2009).”
The COGs (context, object, and goals) in multisensory processing (2016, background): “…(in)congruence of object semantic features in multisensory stimuli starting at approximately 150–200 ms post-stimulus (evoked responses: Raij et al. 2000; Molholm et al. 2004; Diaconescu et al. 2011; but see Naci et al. 2012 for effects <100 ms; induced responses: Yuval-Greenberg and Deouell 2007).”
Neurophysiological and Computational Principles of Cortical Rhythms in Cognition (2010, background): “Recent experimental evidence, however, points to the possibility that multisensory interplay involves coherent oscillations across cortical areas (349, 490, 549, 551, 627, 687, 857, 1087).”
The role of auditory cortices in the retrieval of single‐trial auditory–visual object memories (2015, unknown): “Alternatively, multisensory representations may be predominantly located within visual cortices or rely on their co-activation with auditory regions, given the predominance of vision in object memories (Yuval-Greenberg & Deouell, 2007, 2009; Cohen et al., 2009).”