Turn-taking cues in task-oriented dialogue

By Agustin Gravano, Julia Hirschberg. 2011

302 citations recorded by the source corpus.

Abstract: As interactive voice response systems become more prevalent and provide increasingly more complex functionality, it becomes clear that the challenges facing such systems are not solely in their synthesis and recognition capabilities. Issues such as the coordination of turn exchanges between system and user also play an important role in system usability. In particular, both systems and users have difficulty determining when the other is taking or relinquishing the turn. In this paper, we seek to identify turn-taking cues correlated with human-human turn exchanges which are automatically computable. We compare the presence of potential prosodic, acoustic, and lexico-syntactic turn-yielding cues in prosodic phrases preceding turn changes (smooth switches) vs. turn retentions (holds) vs. backchannels in the Columbia Games Corpus, a large corpus of task-oriented dialogues, to determine which features reliably distinguish between these three. We identify seven turn-yielding cues, all of which can be extracted automatically, for future use in turn generation and recognition in interactive voice response (IVR) systems. Testing Duncan's (1972) hypothesis that these turn-yielding cues are linearly correlated with the occurrence of turn-taking attempts, we further demonstrate that, the greater the number of turn-yielding cues that are present, the greater the likelihood that a turn change will occur. We also identify six cues that precede backchannels, which will also be useful for IVR backchannel generation and recognition; these cues correlate with backchannel occurrence in a quadratic manner. We find similar results for overlapping and for non-overlapping speech.

Subfields: Artificial Intelligence, Experimental and Cognitive Psychology, Language and Linguistics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Experimental and Cognitive Psychology

In Language and Linguistics

Citations in the corpus

Generative Spoken Dialogue Language Modeling (2022, background): “Decades-long research on conversation analysis (Duncan, 1972; Sacks et al., 1974; Schegloff, 2000; Gravano and Hirschberg, 2011; Levinson and Torreira, 2015; Ward, 2019) has shown that human turn-taking relies on a variety of complex signals, or cues, including prosodic cues, linguistic cues and…”
Neural Dialogue Context Online End-of-Turn Detection (2018, methodology): “Various methods have been examined for modeling the end-of-turn detection (Koiso et al., 1998; Shriberg et al., 2000; Schlangen, 2006; Gravano and Hirschberg, 2011; Sato et al., 2002; Gun-takandla and Nielsen, 2015; Ferrer et al., 2002, 2003; Atterer et al., 2008; Arsikere et al., 2014, 2015).”
TurnGPT: a Transformer-based Language Model for Predicting Turn-taking in Spoken Dialog (2020, background): “So far, most prediction models of turn-shifts have used a very simplistic account of syntactic completion, such as the ﬁnal part-of-speech tags (Gravano and Hirschberg, 2011; Meena et al., 2014; Johansson and Skantze, 2015).”
Investigating Speech Features for Continuous Turn-Taking Prediction Using LSTMs (2018, unknown): “POS features have been found in previous studies as good turn-taking features [4, 18].”
Data-driven models for timing feedback responses in a Map Task dialogue system (2014, methodology): “…prosodic features such as intonation patterns, speaking rate, and acoustic cues such as jitter, shimmer and noise to harmonic ratio have been also identified as useful behavioural cues for prediction of turn-taking and backchannel relevant places (Koiso et al., 1998; Gravano & Hirschberg, 2011).”
Towards a General, Continuous Model of Turn-taking in Spoken Dialogue using LSTM Recurrent Neural Networks (2017, background): “Studies have also shown that the more turn-yielding cues are presented together, the more likely it is that the other speaker will take the turn (Gravano and Hirschberg, 2011; Koiso et al., 1998; Duncan and Niederehe, 1974).”