Facilitation of multisensory integration by the "unity effect" reveals that speech is special.

By Argiro Vatakis, Asif A. Ghazanfar, Charles Spence. 2008

97 citations recorded by the source corpus.

Whenever two or more sensory inputs are highly consistent in one or more dimension(s), observers will be more likely to perceive them as a single multisensory event rather than as separate unimodal events. For audiovisual speech, but not for other noncommunicative events, participants exhibit a "unity effect," whereby they are less sensitive to temporal asynchrony (i.e., that are more likely to bind the multisensory signals together) for matched (than for mismatched) speech events. This finding suggests that the modulation of multisensory integration by the unity effect in humans may be specific to speech. To test this hypothesis directly, we investigated whether the unity effect would also influence the multisensory integration of vocalizations from another primate species, the rhesus monkey. Human participants made temporal order judgments for both matched and mismatched audiovisual stimuli presented at a range of stimulus-onset asynchronies. The unity effect was examined with (1) a single call-type across two different monkeys, (2) two different call-types from the same monkey, (3) human versus monkey "cooing," and (4) speech sounds produced by a male and a female human. The results show that the unity effect only influenced participants' performance for the speech stimuli; no effect was observed for monkey vocalizations or for the human imitations of monkey calls. These findings suggest that the facilitation of multisensory integration by the unity effect is specific to human speech signals.

Subfields: Developmental Biology, Experimental and Cognitive Psychology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Developmental Biology

In Experimental and Cognitive Psychology

Citations in the corpus

Assessing the Role of the ‘Unity Assumption’ on Multisensory Integration: A Review (2017, methodology): “…case being a person playing a note on a piano vs. on a guitar), object actions (someone smashing a block of ice or FIGURE 6 | An example of stimuli and results of Vatakis and colleagues’ experiments of the unity effect on temporal perception (Vatakis and Spence, 2007, 2008; Vatakis et al., 2008).”
Perception of intersensory synchrony: A tutorial review (2010, unknown): “Navarra et al. (2005) and Vatakis et al. (2008) also tested generalization for audio–visual temporal recalibration using stimuli from different domains (speech/nonspeech).”
Perception of intersensory synchrony in audiovisual speech: not that special. (2011, background): “More recently, though, Vatakis, Ghazanfar, and Spence (2008) qualified these findings and reported that the congruency effect may be specific for human audiovisual speech, because there was no difference between matching or mismatching call-types of monkeys (‘‘cooing” versus ‘‘grunt” or ‘‘threat”…”
The Curious Incident of Attention in Multisensory Integration: Bottom-up vs. Top-down (2016, unknown): “On the basis of the latter studies, it has been argued that audio- visual speech perception is a special form of multisensory processing (Tuomainen et al., 2005; Vatakis et al, 2008; ) (cf. Vroomen and Stekelenburg, 2011).”
No effect of synesthetic congruency on temporal ventriloquism (2010, background): “Also, others who did use meaningful and complex stimuli, nevertheless failed to observe congruency effects (Koppen, Alsius, & Spence, 2008; Vatakis et al., 2008; Vatakis & Spence, 2006).”
Heterochrony and Cross-Species Intersensory Matching by Infant Vervet Monkeys (2009, background): “thus, readily recognize the correspondence between these very same rhesus monkey expressions [23].”
Intersensory binding across space and time: A tutorial review (2013, background): “…grouping precedes intersensory binding Keetels et al. (2007) AV-TOJ Role of cross-modal correspondence Poor JNDs for audiovisual congruent speech Vatakis et al. (2008) AV-TOJ Role of cross-modal correspondence Poor JNDs for congruent pitch–size Parise and Spence (2009) AV-TOJ Role of…”