On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detection

By Vivian Lai, Chenhao Tan. 2018

469 citations recorded by the source corpus.

Humans are the final decision makers in critical tasks that involve ethical and legal concerns, ranging from recidivism prediction, to medical diagnosis, to fighting against fake news. Although machine learning models can sometimes achieve impressive performance in these tasks, these tasks are not amenable to full automation. To realize the potential of machine learning for improving human decisions, it is important to understand how assistance from machine learning models affects human performance and human agency. In this paper, we use deception detection as a testbed and investigate how we can harness explanations and predictions of machine learning models to improve human performance while retaining human agency. We propose a spectrum between full human agency and full automation, and develop varying levels of machine assistance along the spectrum that gradually increase the influence of machine predictions. We find that without showing predicted labels, explanations alone slightly improve human performance in the end task. In comparison, human performance is greatly improved by showing predicted labels (>20% relative improvement) and can be further improved by explicitly suggesting strong machine performance. Interestingly, when predicted labels are shown, explanations of machine predictions induce a similar level of accuracy as an explicit statement of strong machine performance. Our results demonstrate a tradeoff between human performance and human agency and show that explanations of machine predictions can moderate this tradeoff.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Should We Trust Algorithms? (2020, methodology): “In an experimental study of how much an algorithm adds to human accuracy , Lai & Tan (2019) found that providing an individualized explanation added as much as providing a generic assurance about the algorithm’s quality.”
Learning to Deceive with Attention-Based Explanations (2019, background): “Similar claims about attention’s utility for interpreting models’ predictions are common in the literature (Li et al., 2016; Xu et al., 2015; Choi et al., 2016; Xie et al., 2017; Martins and Astudillo, 2016; Lai and Tan, 2019).”
To Trust or to Think (2021, background): “Consistent with prior research [7, 9, 26, 41], our results demonstrate that people aided by simple explainable AI interfaces performed better overall than people who completed the task without any AI support.”
Explanation-Based Human Debugging of NLP Models: A Survey (2021, background): “Lakkaraju et al. (2020) remarked that the evaluation setup of existing EBHD work is often too easy or unrealistic.”
Tree Space Prototypes: Another Look at Making Tree Ensembles Interpretable (2016, result): “We note that self-reported user confidence has not been found to be indicative of actual performance and can sometimes even be misleading [15, 27], with studies finding that humans cannot accurately assess their own performance [26, 42].”
Harnessing Explanations to Bridge AI and Humans (2020, background): “We are not aware of any such experiment that has reported a significant improvement in accuracy that cannot be explained by increased subject trust in a model whose accuracy is higher than the human baseline (such as Lai and Tan 2019 [11]).”
Single-Turn Debate Does Not Help Humans Answer Hard Reading-Comprehension Questions (2022, background): “, 2018), data annotation (Schmidt and Biessmann, 2019) and deception detection (Lai and Tan, 2019).”
The Principles and Limits of Algorithm-in-the-Loop Decision Making (2019, background): “Although people appear in some contexts to follow correct predictions more than incorrect ones [48], other studies suggest that people are unable to distinguish between reliable and unreliable predictions [33] or to detect algorithmic errors [62].”
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance (2020, background): “ge-scale study provides informative insights that will inform future work on these domains. Finally, we only explored a small fraction of possible ways to present explanations: highlighting key words [18, 36, 42, 62] and natural language arguments. It may be that alternative approaches provide more ”
Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies (2021, methodology): “, LSAT questions [13], hotel reviews [80, 81], etc.”
Proxy tasks and subjective measures can be misleading in evaluating explainable AI systems (2020, background): “n [16] discovered that self-reported measures could be misleading, since participant’s con•dence in their performance was negatively associated with their actual performance. Similarly, Lai & Tan [28] found that humans cannot accurately estimate their own performance. More closely related to our”
Are Explanations Helpful? A Comparative Study of the Effects of Explanations in AI-Assisted Decision-Making (2021, background): “Lai and Tan [42] deception detection feature contribution N/A N/A ✓ ?”
Semi-Automated Data Labeling (2020, background): “In a study of human-machine decision making, Lai and Tan (2019) demonstrated that providing humans with machine predictions signiﬁcantly improved human decision-making performance in a deception-detection task.”
Evaluating and Characterizing Human Rationales (2020, methodology): “ERASER (DeYoung et al., 2019) includes three additional datasets: CoS-( Lai and Tan, 2019; Poursabzi-Sangdeh et al., 2018; Lage et al., 2018; Lai et al., 2020).”
Two-Turn Debate Doesn't Help Humans Answer Hard Reading Comprehension Questions (2022, background): “Previous studies have reported that model-generated explanations can aid humans in some tasks (Cai et al., 2019; Lundberg et al., 2018; Schmidt and Biessmann, 2019; Lai and Tan, 2019), though only when the models are generally able to outperform humans at that task (Bansal et al.”
Explanations Can Reduce Overreliance on AI Systems During Decision-Making (2022, background): “Prior XAI literature has used a variety of tasks, including binary classification [3, 4, 8, 54] and question answering about images [12, 15] and text [4, 26].”
Debugging Tests for Model Explanations (2020, background): “In a cooperative setting, Lai and Tan [34] find that the humans exploit label information and Feng and Boyd-Graber [22] demonstrate how to assess explanations in a natural language setting.”
Understanding the Effect of Out-of-distribution Examples and Interactive Explanations on Human-AI Decision Making (2021, methodology): “Second, although explaining AI predictions has been hypothesized to help humans understand AI predictions and thus improve human performance [13], static explanations, such as highlighting important features and showing AI confidence, have been mainly explored so far [2, 21, 33].”