Towards a General, Continuous Model of Turn-taking in Spoken Dialogue using LSTM Recurrent Neural Networks
By Gabriel Skantze. 2017
104 citations recorded by the source corpus.
Previous models of turn-taking have mostly been trained for specific turn-taking decisions, such as discriminating between turn shifts and turn retention in pauses. In this paper, we present a predictive, continuous model of turn-taking using Long Short-Term Memory (LSTM) Recurrent Neural Networks (RNN). The model is trained on human-human dialogue data to predict upcoming speech activity in a future time window. We show how this general model can be applied to two different tasks that it was not specifically trained for. First, to predict whether a turn-shift will occur or not in pauses, where the model achieves a better performance than human observers, and better than results achieved with more traditional models. Second, to make a prediction at speech onset whether the utterance will be a short backchannel or a longer utterance. Finally, we show how the hidden layer in the network can be used as a feature vector for turn-taking decisions in a human-robot interaction scenario.
Subfields: Artificial Intelligence, Experimental and Cognitive Psychology, Language and Linguistics
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Experimental and Cognitive Psychology
In Language and Linguistics
Citations in the corpus
- Generative Spoken Dialogue Language Modeling (2022, neutral): “More recently, more sophisticated machine learning-based models of turn-taking have been introduced (Meena et al., 2014; Skantze, 2017; Roddy et al., 2018; Masumura et al., 2018).”
- Neural Dialogue Context Online End-of-Turn Detection (2018, background): “…sequential features without manual specification of fixed length features such as maximum, minimum, average values of acoustic features or bag-of-words features (Masumura et al., 2017; Skantze, 2017) We note, however, that interlocutor’s utterances are rarely used for end-of-turn detection.”
- TurnGPT: a Transformer-based Language Model for Predicting Turn-taking in Spoken Dialog (2020, methodology): “However, we argue that previous models that have combined linguistic and prosodic cues (cf. Meena et al. 2014; Skantze 2017; Roddy et al. 2018; Masumura et al. 2019) have used too simplistic models of linguistic turn-constructional units.”
- Investigating Speech Features for Continuous Turn-Taking Prediction Using LSTMs (2018, unknown): “In this paper we significantly extend the original work of Skantze in [10].”