Designing Contestability: Interaction Design, Machine Learning, and Mental Health

By Tad Hirsch, Kritzia Merced, Shrikanth S. Narayanan, Zac E. Imel, David C. Atkins. 2017

119 citations recorded by the source corpus.

We describe the design of an automated assessment and training tool for psychotherapists to illustrate challenges with creating interactive machine learning (ML) systems, particularly in contexts where human life, livelihood, and wellbeing are at stake. We explore how existing theories of interaction design and machine learning apply to the psychotherapy context, and identify "contestability" as a new principle for designing systems that evaluate human behavior. Finally, we offer several strategies for making ML systems more accountable to human actors.

Subfields: Applied Psychology, Clinical Psychology, Experimental and Cognitive Psychology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Applied Psychology

1. Mark O. Riedl
2. Brent Harrison
3. Upol Ehsan

In Clinical Psychology

1. Adam S. Miner
2. K. G. Karahalios
3. David C. Atkins
4. Berk Ustun
5. Emily L. Denton
6. A. Hanna

In Experimental and Cognitive Psychology

1. Ben Hutchinson
2. Christina Greer
3. Mark O. Riedl
4. Berk Ustun
5. Emily L. Denton
6. Margaret Mitchell

Citations in the corpus

• Predictive Multiplicity in Classification (2019, background): “…and severity of conflicting predictions allows them to challenge key aspects of modern machine learning [as per the principles of contestability 20, 26] – from model selection (e.g., how should we choose between competing models, if we should choose one at all, and who should have a say in…”
• Automated rationale generation: a technique for explainable AI and its effects on human perceptions (2019, background): “ong the dimension of interactivity for the next iteration. For instance, contestability, the ability to either reject a reasoning or ask for another one, which has shown to improve user satisfactions [22,16] can be incorporated in the future. Second, our data collection pipeline is currently designe”
• Contestability in Algorithmic Systems (2019, background): “introduced contestability as a design objective [5].”
• Towards Accountability for Machine Learning Datasets: Practices from Software Engineering and Infrastructure (2020, background): “been motivated as a way of encouraging critical and responsible engagement between various stakeholders, surfacing values that might not otherwise be visible, and safeguarding a system against misuse [68, 103]. However, the ability of a given stakeholder to challenge an automated decision is limited”
• Interpretable to Whom? A Role-based Model for Analyzing Interpretable Machine Learning Systems (2018, background): “If the decision-subject is not also the executor, then they will want to know why an executor made a particular decision, either out of plain curiosity, or to be able to challenge or change that decision — see (Hirsch et al., 2017) for an exploration of this idea of contestability .”
• Human–AI collaboration enables more empathic conversations in text-based peer-to-peer mental health support (2022, background): “…between expressed empathy and positive For AI-assisted writing, for instance, we must build AI models that generate actionable writing suggestions and simultaneously design human-facing systems that help people see, understand and act on those suggestions in a just-in-time fashion 17–23 .”