Predicting Student Emotions in Computer-Human Tutoring Dialogues
By Diane J. Litman, Katherine Forbes-Riley. 2004
210 citations recorded by the source corpus.
We examine the utility of speech and lexical features for predicting student emotions in computer-human spoken tutoring dialogues. We first annotate student turns for negative, neutral, positive and mixed emotions. We then extract acoustic-prosodic features from the speech signal, and lexical items from the transcribed or recognized speech. We compare the results of machine learning experiments using these features alone or in combination to predict various categorizations of the annotated student emotions. Our best results yield a 19-36% relative improvement in error reduction over a baseline. Finally, we compare our results with emotion prediction in human-human tutoring dialogues.
Subfields: Artificial Intelligence, Social Psychology, Dialogue & Conversational AI, Sentiment & Opinion Analysis
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Saif M. Mohammad
- S. D’Mello
- Yoav Goldberg
- Mohit Bansal
- A. Graesser
- Jianfeng Gao
- Mirella Lapata
- Yejin Choi
In Social Psychology
- François Mairesse
- Marilyn A. Walker
- Saif M. Mohammad
- S. D’Mello
- Cecilia Ovesdotter Alm
- Justine Cassell
- William Yang Wang
- Peter D. Turney
In Dialogue & Conversational AI
- Marilyn A. Walker
- François Mairesse
- S. D’Mello
- Saif M. Mohammad
- A. Graesser
- Verena Rieser
- Oliver Lemon
In Sentiment & Opinion Analysis
Citations in the corpus
- Controlling User Perceptions of Linguistic Style: Trainable Generation of Personality Traits (2011, background): “…systems such as intelligent tutoring systems are more effective if they can generate a range of different types of stylistic linguistic variation (Litman and Forbes-Riley 2004, 2006; Porayska-Pomsta and Mellish 2004; Wang et al. 2005; McQuiggan, Mott, and Lester 2008; Tapus and Mataric 2008).”
- Using Hashtags to Capture Fine Emotion Categories from Tweets (2015, unknown): “, 2006), and automatic tutoring systems (Litman and Forbes-Riley, 2004).”
- Multimodal Affective Analysis Using Hierarchical Attention Strategy with Word-Level Alignment (2018, background): “Early work combined human transcribed lexical features and low-level hand-crafted acoustic features using feature-level fusion (Forbes-Riley and Litman, 2004; Litman and Forbes-Riley, 2004).”
- Emotions from Text: Machine Learning for Text-based Emotion Prediction (2005, background): “Whereas work on emotion classification from the point of view of natural speech and human-computer dialogues is fairly extensive, e.g. (Scherer, 2003), (Litman and Forbes-Riley, 2004), this appears not to be the case for text-to-speech synthesis (TTS).”
- “Love ya, jerkface”: Using Sparse Log-Linear Models to Build Positive and Impolite Relationships with Teens (2012, background): “Although speaker state prediction has attracted much attention in the dialogue research community, most studies have focused on the analysis of anger, frustration, and other classic emotions (Litman and Forbes-Riley, 2004; Liscombe et al., 2005; Devillers and Vidrascu, 2006; Ai et al., 2006; Grimm e”
- Empirically building and evaluating a probabilistic model of user affect (2009, background): “…Kapoor and Picard 2005, Healey and Picard 2005), lower–level affective measures such as affective valence and arousal (e.g., Prendinger et al. 2005, Litman and Forbes-Riley 2004) or overall emotional predisposition over a complete interaction (Mandryk et al. 2006, Yannakakis et al. 2008) Other…”
- CROWDSOURCING A WORD–EMOTION ASSOCIATION LEXICON (2013, background): “There is some support for the hypothesis that students learn better and faster when they are in a positive emotional state (Litman and Forbes-Riley, 2004).”
- Multimodal semi-automated affect detection from conversational cues, gross body language, and facial features (2010, background): “…can be considered to be a separate modality from lexical features derived by transcribing the speech. involves a combination of acoustic-prosodic, lexical, and discourse features for affect detection (Ang et al. 2002; Lee and Narayanan 2005; Liscombe et al. 2005; Litman and Forbes-Riley 2004).”
- Features and classifiers for emotion recognition from speech: a survey from 2000 to 2011 (2012, background): “Similarly, Litman and Forbes-Riley (2004) examine the utility of speech and lexical features for predicting student emotions in computer-human tutoring dialogues.”
- Better to be frustrated than bored: The incidence, persistence, and impact of learners' cognitive-affective states during interactions with three different computer-based learning environments (2010, background): “than the kappa for usage observations, it is on par with kappas reported by other researchers who have assessed the reliability of detecting naturally occurring emotional expressions (Ang et al., 2002; Grimm et al., 2006; Litman and Forbes-Riley, 2004; Shafran et al., 2003).”
- A Review and Meta-Analysis of Multimodal Affect Detection Systems (2015, background): “[Litman and Forbes-Riley 2004] 15 nat dim class 3 dim simple Voice + Text feat dep”