‘When Birds of a Feather Flock Together’: Synesthetic Correspondences Modulate Audiovisual Integration in Non-Synesthetes

By Cesare V. Parise, Charles Spence. 2009

266 citations recorded by the source corpus.

Background Synesthesia is a condition in which the stimulation of one sense elicits an additional experience, often in a different (i.e., unstimulated) sense. Although only a small proportion of the population is synesthetic, there is growing evidence to suggest that neurocognitively-normal individuals also experience some form of synesthetic association between the stimuli presented to different sensory modalities (i.e., between auditory pitch and visual size, where lower frequency tones are associated with large objects and higher frequency tones with small objects). While previous research has highlighted crossmodal interactions between synesthetically corresponding dimensions, the possible role of synesthetic associations in multisensory integration has not been considered previously. Methodology Here we investigate the effects of synesthetic associations by presenting pairs of asynchronous or spatially discrepant visual and auditory stimuli that were either synesthetically matched or mismatched. In a series of three psychophysical experiments, participants reported the relative temporal order of presentation or the relative spatial locations of the two stimuli. Principal Findings The reliability of non-synesthetic participants' estimates of both audiovisual temporal asynchrony and spatial discrepancy were lower for pairs of synesthetically matched as compared to synesthetically mismatched audiovisual stimuli. Conclusions Recent studies of multisensory integration have shown that the reduced reliability of perceptual estimates regarding intersensory conflicts constitutes the marker of a stronger coupling between the unisensory signals. Our results therefore indicate a stronger coupling of synesthetically matched vs. mismatched stimuli and provide the first psychophysical evidence that synesthetic congruency can promote multisensory integration. Synesthetic crossmodal correspondences therefore appear to play a crucial (if unacknowledged) role in the multisensory integration of auditory and visual information.

Subfields: Cognitive Neuroscience, Experimental and Cognitive Psychology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Experimental and Cognitive Psychology

Citations in the corpus

Assessing the Role of the ‘Unity Assumption’ on Multisensory Integration: A Review (2017, background): “The results of the three experiments reported by Parise and Spence (2009) are consistent with the view that more pronounced multisensory integration occurs for congruent visual and auditory stimuli than for those pairs of stimuli that happen to be incongruent.”
No effect of synesthetic congruency on temporal ventriloquism (2010, background): “In line with the idea that synesthetic congruency affects intersensory binding, Parise and Spence (2009) reported that sensitivity for the AV temporal order of synesthetically congruent pairs (i.e., a high-pitched tone with a small-sized visual stimulus or a low-pitched tone with a large-sized…”
Intersensory binding across space and time: A tutorial review (2013, background): “…Morein-Zamir et al. (2003) Visual TOJ Role of spatial discordance No effect of audiovisual spatial discordance Vroomen and Keetels (2006) Visual TOJ Role of auditory grouping Auditory grouping precedes intersensory binding Keetels et al. (2007) AV-TOJ Role of cross-modal correspondence Poor…”
Managing sensory expectations concerning products and brands: Capitalizing on the potential of sound and shape symbolism (2012, background): “One ubiquitous crossmodal correspondence is that between larger objects (no matter whether seen or felt) and lower-pitched sounds, and smaller objects and higher-pitched sounds (Parise & Spence, 2009; Walker & ⁎ Fax: +44 1865 310447.”
"Bouba" and "Kiki" in Namibia? A remote culture make similar shape-sound matches, but different shape-taste matches to Westerners. (2013, background): “…enjoyed particular attention given that the ability to make such links may help in localising stimuli (Bien, ten Oever, Goebel, & Sack, 2012; Parise & Spence, 2009), and in solving the ‘‘crossmodal correspondence problem’’ (i.e., the problem of how to know which stimuli to link together…”
The effect of synesthetic associations between the visual and auditory modalities on the Colavita effect (2015, background): “The task that induces the Colavita effect and other tasks measuring audiovisual integration (Keetels and Vroomen 2007; Parise and Spence 2008, 2009; Bien et al. 2012; Klapetek et al. 2012) minimize the influence of decision/response selection.”
Cross-correlation between auditory and visual signals promotes multisensory integration. (2013, methodology): “The stimuli came from discrepant locations along the horizontal axis, and, using the same experimental paradigm used by Parise and Spence (2009), observers had to report the left/right location of the auditory stimulus with respect to the visual stimulus (see Fig.”
The COGs (context, object, and goals) in multisensory processing (2016, background): “On the one hand, judgements of sound localisation (left/right) appear more erroneous if visual stimuli accompanying the sounds match the “intuitive” pitch–size association (Parise and Spence 2009).”
The Brain’s Tendency to Bind Audiovisual Signals Is Stable but Not General (2016, unknown): “…tendency observed in the present study is consistent with the stability and task specificity reported else-where for synesthesia, which could be interpreted as an extreme, unique manifestation of the binding tendency (Ghazanfar & Schroeder, 2006; Newell & Mitchell, 2015; Parise & Spence, 2009).”
When correlation implies causation in multisensory integration. (2012, unknown): “Previous research has demonstrated that spatiotemporal cues, alongwith prior knowledge, are exploited by the human brain to solve this problem [4–9].”
Does crossmodal correspondence modulate the facilitatory effect of auditory cues on visual search? (2012, background): “…between auditory pitch and visual brightness were to be semantic in nature, it could possibly never manifest itself at a preattentive stage of perceptual processing (see also Gallace & Spence, 2006), while a structural or statistical correspondence might (see Ernst, 2007; Parise & Spence, 2009).”
That sounds sweet: using cross-modal correspondences to communicate gustatory attributes (2015, background): “Extant laboratory research in cognitive psychology and sensory neuroscience demonstrates enhanced multisensory integration of those sensory features that correspond crossmodally (see Parise & Spence, 2009; Spence, 2011, for a review).”