Multimodal semi-automated affect detection from conversational cues, gross body language, and facial features

By S. D’Mello, A. Graesser. User modeling and user-adapted interaction, 2010

346 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Cognitive Neuroscience, Experimental and Cognitive Psychology

View Paper: https://www.semanticscholar.org/paper/faaf4285ce136620f0ccecb863e515c2f7dc580a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

In Experimental and Cognitive Psychology

Citations in the corpus

Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications (2010, background): “D’Mello and Graesser [177] considered a combination of facial features, gross body language, and conversational cues for detecting some of the learning-centered affective states.”
AutoTutor and affective autotutor: Learning by talking with cognitively and emotionally intelligent computers that talk back (2012, methodology): “1 and [D'Mello and Graesser 2010] for additional methodological details).”
Emotional design in multimedia learning. (2012, background): “…in research on anthropomorphism , which studies the attribution of uniquely human characteristics and qualities to nonhuman beings, inanimate objects, or natural or supernatural phenomena (Dehn & van Mulken, 2000; Disalvo & Gemperle, 2007; Hongpaisanwiwat & Lewis, 2003; Reeves, & Nass, 1996).”
Gaze tutor: A gaze-reactive intelligent tutoring system (2012, background): “Consistent with predictions of the forced-effort model (Larson and Richards, 1991), bored students are more likely to transition into frustration (D’Mello and Graesser, 2010a).”
AutoTutor and Family: A Review of 17 Years of Natural Language Tutoring (2014, methodology): “More recent systems have also used this information as part of more advanced student models, such as models that estimate affective states (D’Mello and Graesser 2010) and persistent student models (Nye et al.”
The Faces of Engagement: Automatic Recognition of Student Engagementfrom Facial Expressions (2014, methodology): “Our work extends prior research on engagement recognition using computer vision [10], [11], [29], [42] and is arguably the most thorough study on this topic to date: We collected a data set of student facial expressions while performing a cognitive training task.”
Dynamics of affective states during complex learning (2012, methodology): “Third, the offline affect annotations obtained via this retrospective protocol correlate with online recordings of facial activity and gross body movements in expected directions (D’Mello & Graesser, 2010).”
Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition: History, Trends, and Affect-related Applications (2016, methodology): “In [84], a correspondence is established manually between landmarks of the model and the captured data, using a Thin Plate Spline (TPS) model to deform the shape.”