Robust speech perception: Recognize the familiar, generalize to the similar, and adapt to the novel

By Dave F. Kleinschmidt, T. Florian Jaeger. Psychology Review, 2015

572 citations recorded by the source corpus.

Successful speech perception requires that listeners map the acoustic signal to linguistic categories. These mappings are not only probabilistic, but change depending on the situation. For example, one talker’s /p/ might be physically indistinguishable from another talker’s /b/ (cf. lack of invariance). We characterize the computational problem posed by such a subjectively non-stationary world and propose that the speech perception system overcomes this challenge by (1) recognizing previously encountered situations, (2) generalizing to other situations based on previous similar experience, and (3) adapting to novel situations. We formalize this proposal in the ideal adapter framework: (1) to (3) can be understood as inference under uncertainty about the appropriate generative model for the current talker, thereby facilitating robust speech perception despite the lack of invariance. We focus on two critical aspects of the ideal adapter. First, in situations that clearly deviate from previous experience, listeners need to adapt. We develop a distributional (belief-updating) learning model of incremental adaptation. The model provides a good fit against known and novel phonetic adaptation data, including perceptual recalibration and selective adaptation. Second, robust speech recognition requires listeners learn to represent the structured component of cross-situation variability in the speech signal. We discuss how these two aspects of the ideal adapter provide a unifying explanation for adaptation, talker-specificity, and generalization across talkers and groups of talkers (e.g., accents and dialects). The ideal adapter provides a guiding framework for future investigations into speech perception and adaptation, and more broadly language comprehension.

Subfields: Artificial Intelligence, Experimental and Cognitive Psychology, Signal Processing

View Paper: https://www.semanticscholar.org/paper/6543ff5d6b7517fcd8fb1bc1449ad09dd5a750c6

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Experimental and Cognitive Psychology

In Signal Processing

Citations in the corpus

The cave of Shadows. Addressing the human factor with generalized additive mixed models (2015, background): “l features and object names are subject to continuous updating. Ramscar et al. (2010) and Arnon and Ramscar (2012) documented the consequences of within-experiment learning in the domain of language. Kleinschmidt and Jaeger (2015) report and model continuous updating in auditory processing in the co”
Learning (not) to predict: Grammatical gender processing in second language acquisition (2016, background): “…mirror findings from speech perception, where listeners rapidly adapt their prediction according to speaker identity and implicitly learn to adjust their expectations from previous prediction errors (e.g. Kraljic and Samuel, 2007; Kleinschmidt and Jaeger, 2015; for syntax, see Kamide, 2012).”
What do we mean by prediction in language comprehension? (2015, background): “…processing (e.g. Bejjanki et al., 2011; Chater, Crocker, & Pickering, 1998; Clayards, Tanenhaus, Aslin, & Jacobs, 2008; Feldman et al., 2009; Kleinschmidt & Jaeger, 2015; Levy, 2008; Norris, 2006; Norris & McQueen, 2008; see also Crocker & Brants, 2000; Hale, 2001; Jurafsky, 1996; Narayanan…”
Continual Adaptation for Efficient Machine Communication (2019, background): “Adapting or personalizing language models is a classic prob- lem of practical interest for NLP, where shifts in the data distribution are often found across test contexts (Kneser and Steinbiss, 1993; Riccardi and Gorin, 2000; Bellegarda, 2004; Ben-David et al., 2010).”