Understanding the Effect of Out-of-distribution Examples and Interactive Explanations on Human-AI Decision Making

By Han Liu, Vivian Lai, Chenhao Tan. 2021

149 citations recorded by the source corpus.

Although AI holds promise for improving human decision making in societally critical domains, it remains an open question how human-AI teams can reliably outperform AI alone and human alone in challenging prediction tasks (also known as complementary performance). We explore two directions to understand the gaps in achieving complementary performance. First, we argue that the typical experimental setup limits the potential of human-AI teams. To account for lower AI performance out-of-distribution than in-distribution because of distribution shift, we design experiments with different distribution types and investigate human performance for both in-distribution and out-of-distribution examples. Second, we develop novel interfaces to support interactive explanations so that humans can actively engage with AI assistance. Using virtual pilot studies and large-scale randomized experiments across three tasks, we demonstrate a clear difference between in-distribution and out-of-distribution, and observe mixed results for interactive explanations: while interactive explanations improve human perception of AI assistance's usefulness, they may reinforce human biases and lead to limited performance improvement. Overall, our work points out critical challenges and future directions towards enhancing human performance with AI assistance.

Subfields: Artificial Intelligence, Health Informatics, Safety Research, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Health Informatics

Q. Liao

In Safety Research

In Safety, Robustness & Interpretability

Citations in the corpus

Measuring Progress on Scalable Oversight for Large Language Models (2022, background): “…with the literature on human–AI team decision making, surveyed in Lai et al. (2021), and including targeted case studies on recidivism prediction (e.g. Liu et al., 2021), medical diagnosis (e.g. Lakkaraju et al., 2016), and credit risk prediction (e.g. Chromik et al., 2021), among many others.”
Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies (2021, methodology): ““Shallow” models Logistic regression [16, 39, 41, 45, 68, 86, 106, 143], linear regression [28, 37, 101, 111], generalized additive models∗ (GAMs) [1, 13, 39, 43, 49, 128, 135] decision trees/random forests [29, 45, 54, 55, 86, 92, 97, 137, 144, 155], support-vector machines (SVMs) [41, 80, 81, 86, ”
When combinations of humans and AI are useful: A systematic review and meta-analysis (2024, unknown): “But some studies show that augmenting humans with AI can lead to better outcomes than humans or AI working alone 24–26 , while others show the opposite 4,7,9 .”