Personalized machine learning for robot perception of affect and engagement in autism therapy

By Ognjen Rudovic, Jaeryoung Lee, Miles Dai, Björn Schuller, Rosalind W. Picard. Science Robotics, 2018

311 citations recorded by the source corpus.

Personalized machine learning enables robot perception of children’s affective states and engagement during robot-assisted autism therapy. Robots have the potential to facilitate future therapies for children on the autism spectrum. However, existing robots are limited in their ability to automatically perceive and respond to human affect, which is necessary for establishing and maintaining engaging interactions. Their inference challenge is made even harder by the fact that many individuals with autism have atypical and unusually diverse styles of expressing their affective-cognitive states. To tackle the heterogeneity in children with autism, we used the latest advances in deep learning to formulate a personalized machine learning (ML) framework for automatic perception of the children’s affective states and engagement during robot-assisted autism therapy. Instead of using the traditional one-size-fits-all ML approach, we personalized our framework to each child using their contextual information (demographics and behavioral assessment scores) and individual characteristics. We evaluated this framework on a multimodal (audio, video, and autonomic physiology) data set of 35 children (ages 3 to 13) with autism, from two cultures (Asia and Europe), and achieved an average agreement (intraclass correlation) of ~60% with human experts in the estimation of affect and engagement, also outperforming nonpersonalized ML solutions. These results demonstrate the feasibility of robot perception of affect and engagement in children with autism and have implications for the design of future autism therapies.

Subfields: Cognitive Neuroscience, Experimental and Cognitive Psychology, Social Psychology

View Paper: https://www.semanticscholar.org/paper/2567a27bca06d4c409a1a603281cb78e68f233b1

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

1. Sinan Kalkan
2. Hatice Gunes
3. Nikhil Churamani

In Experimental and Cognitive Psychology

1. Hatice Gunes
2. Björn Schuller
3. Maja Pantic

In Social Psychology

1. Sinan Kalkan
2. Hatice Gunes
3. Nikhil Churamani
4. Björn Schuller

Citations in the corpus

• Dimensions of artificial intelligence anxiety based on the integrated fear acquisition theory (2020, unknown): “The widespread use of AI in healthcare could also displace many healthcare practitioners and an estimated 80% of doctors [62,63].”
• Lifelong Learning and Personalization in Long-Term Human-Robot Interaction (LEAP-HRI) (2021, unknown): “, [5, 10, 23, 25, 27]), retail and customer service (e.”
• FADL: Federated-Autonomous Deep Learning for Distributed Electronic Health Record (2018, background): “The balance between global and local learning is an important factor to consider when designing distributed machine learning methods,especially on health data[14, 19].”
• Continual Learning for Affective Robotics: Why, What and How? (2020, background): “Existing approaches: Most personalised affect perception approaches focus on contextual attributions for each user [53], perform selective weighting of subject-specific data [17], or apply unsupervised clustering of person-specific feature representations [56] to adapt to individual users.”