Towards Facilitating Empathic Conversations in Online Mental Health Support: A Reinforcement Learning Approach
By Ashish Sharma, Inna Wanyin Lin, Adam S. Miner, David C. Atkins, Tim Althoff. 2021
195 citations recorded by the source corpus.
Online peer-to-peer support platforms enable conversations between millions of people who seek and provide mental health support. If successful, web-based mental health conversations could improve access to treatment and reduce the global disease burden. Psychologists have repeatedly demonstrated that empathy, the ability to understand and feel the emotions and experiences of others, is a key component leading to positive outcomes in supportive conversations. However, recent studies have shown that highly empathic conversations are rare in online mental health platforms. In this paper, we work towards improving empathy in online mental health support conversations. We introduce a new task of empathic rewriting which aims to transform low-empathy conversational posts to higher empathy. Learning such transformations is challenging and requires a deep understanding of empathy while maintaining conversation quality through text fluency and specificity to the conversational context. Here we propose Partner, a deep reinforcement learning (RL) agent that learns to make sentence-level edits to posts in order to increase the expressed level of empathy while maintaining conversation quality. Our RL agent leverages a policy network, based on a transformer language model adapted from GPT-2, which performs the dual task of generating candidate empathic sentences and adding those sentences at appropriate positions. During training, we reward transformations that increase empathy in posts while maintaining text fluency, context specificity, and diversity. Through a combination of automatic and human evaluation, we demonstrate that Partner successfully generates more empathic, specific, and diverse responses and outperforms NLP methods from related tasks such as style transfer and empathic dialogue generation. This work has direct implications for facilitating empathic conversations on web-based platforms.
Subfields: Artificial Intelligence, Social Psychology, Reinforcement Learning, Dialogue & Conversational AI
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yejin Choi
- Yulia Tsvetkov
- Tat-seng Chua
- Mohit Bansal
- Diyi Yang
- Xiangnan He
- Maarten Sap
- Ronan Le Bras
In Social Psychology
- Yulia Tsvetkov
- Inna Wanyin Lin
- Ashish Sharma
- Tim Althoff
- Adam S. Miner
- David C. Atkins
- Yejin Choi
- Anind K. Dey
In Reinforcement Learning
- Maarten de Rijke
- Anind K. Dey
- Tat-seng Chua
- Yulia Tsvetkov
- Yejin Choi
- Diyi Yang
- Maarten Sap
- Chongming Gao
In Dialogue & Conversational AI
Citations in the corpus
- Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMs (2022, background): “empathy, interlocutor perspective, and social intelligence (Kearns et al., 2020; Sharma et al., 2021).”
- Gendered Mental Health Stigma in Masked Language Models (2022, methodology): “…about mental health conditions, we have models fill in the masked token, then examine the probabilities of generated words with gender associations. employed on mental health-related tasks (Chancel-lor and De Choudhury, 2020; Sharma et al., 2021, 2022; Zhang and Danescu-Niculescu-Mizil, 2020).”
- Can Large Language Models Transform Computational Social Science? (2023, background): “…triggers for empathetic responses (Buechel et al., 2018; Omitaomu et al., 2022), CSS can reciprocally inform the design of empathetic communities (Coulton et al., 2014; Taylor et al., 2019), as well as community-specific tools like counseling dialogue systems (Sharma et al., 2021; Ma et al., 2020).”
- Human–AI collaboration enables more empathic conversations in text-based peer-to-peer mental health support (2022, methodology): “Therefore, we conducted a second, complementary evaluation by applying the theory-based empathy classification model proposed by Sharma et al. 29 , which assigns a score between 0 and 6 to each response and has been validated and used in prior work 47,79–81 .”
- Leveraging Large Language Models for Mental Health Prediction via Online Text Data (2023, background): “…range of opportunities for UbiComp, Human-Computer Interaction (HCI), and mental health communities, such as online public health monitoring systems [33, 62], intelligent assistants for mental counselors and supporters [74, 75], mental-health-aware personal chatbots [4, 28], to just name a few.”
- Advances and Challenges in Conversational Recommender Systems: A Survey (2021, background): “Thus, other metrics reflecting user satisfaction are more suitable in evaluation, such as measuring fluency [17, 142, 46], consistency [53, 96], readability [95], informativeness [74], diversity [104, 78, 57], and empathy [58, 175].”