Prosody-based automatic segmentation of speech into sentences and topics

By Elizabeth Shriberg, Andreas Stolcke, Dilek Z. Hakkani-Tür, Gökhan Tür. 2000

513 citations recorded by the source corpus.

Abstract A crucial step in processing speech audio data for information extraction, topic detection, or browsing/playback is to segment the input into sentence and topic units. Speech segmentation is challenging, since the cues typically present for segmenting text (headers, paragraphs, punctuation) are absent in spoken language. We investigate the use of prosody (information gleaned from the timing and melody of speech) for these tasks. Using decision tree and hidden Markov modeling techniques, we combine prosodic cues with word-based approaches, and evaluate performance on two speech corpora, Broadcast News and Switchboard. Results show that the prosodic model alone performs on par with, or better than, word-based statistical language models – for both true and automatically recognized words in news speech. The prosodic model achieves comparable performance with significantly less training data, and requires no hand-labeling of prosodic events. Across tasks and corpora, we obtain a significant improvement over word-only models using a probabilistic combination of prosodic and lexical information. Inspection reveals that the prosodic models capture language-independent boundary indicators described in the literature. Finally, cue usage is task and corpus dependent. For example, pause and pitch features are highly informative for segmenting news speech, whereas pause, duration and word-based cues dominate for natural conversation.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

An empirical investigation of sparse log-linear models for improved dialogue act classification (2013, background): “In addition to lexical features, prosodic and syntactic features [11, 12, 13, 14] have also been studied.”
Indeterminacy in language acquisition: the role of child directed speech and joint attention (2004, background): “A classical example is the talking heads study of Steels and Kaplan (2002) in which thousands of agents engaged in discrimination games involving two agents and form of joint visual attention that restricted their shared visual context. In the discrimination game, one agent would describe a physical”
PROSODY MODELS FOR CONVERSATIONAL SPEECH RECOGNITION (2003, methodology): “algorithm using multiple decision trees [35] or by principa l components analysis [33].”
Enriching speech recognition with automatic detection of sentence boundaries and disfluencies (2006, methodology): “We compute prosodic features based on duration, fundamental frequency (F0), energy, and pause information [16].”
Neural Dialogue Context Online End-of-Turn Detection (2018, methodology): “Various methods have been examined for modeling the end-of-turn detection (Koiso et al., 1998; Shriberg et al., 2000; Schlangen, 2006; Gravano and Hirschberg, 2011; Sato et al., 2002; Gun-takandla and Nielsen, 2015; Ferrer et al., 2002, 2003; Atterer et al., 2008; Arsikere et al., 2014, 2015).”
Using Prosodic Features in Language Models for Meetings (2007, background): “Prosody has long been studied as a knowledge source for speech understanding, and has been successfully used for a variety of tasks, including topic segmentation [1], disfluency detection [2], speaker verification [3], and speech recognition [4,5,6].”