The Role of Explanations on Trust and Reliance in Clinical Decision Support Systems

By Adrian Bussone, Simone Stumpf, D. O’Sullivan. 2015 International Conference on Healthcare Informatics, 2015

458 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Health Information Management, Surgery

View Paper: https://www.semanticscholar.org/paper/a8cd3998f50e01bcab7333df1247a55a2ddebefa

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Health Information Management

In Surgery

Citations in the corpus

"Hello AI": Uncovering the Onboarding Needs of Medical Practitioners for Human-AI Collaborative Decision-Making (2019, background): “Domains of study include recommender systems [4], medicine [12, 75], social media [22], creativity [14], and advertisements [23].”
How and What Can Humans Learn from Being in the Loop? (2020, background): “This statement confirms previous literature where medical experts prefer to have better explanations to help them interpret the system’s confidence, to verify that the medical condition fits the suggestion, to better understand the decision model, and to make differential diagnoses [6].”
'It's Reducing a Human Being to a Percentage': Perceptions of Justice in Algorithmic Decisions (2018, background): “In some cases, explanation increases trust and reliance [33, 85, 1, 30, 60], but in others an explanation may have the opposite effect if the level of detail it contains is deemed insufficient [7].”
Social Bias Frames: Reasoning about Social and Power Implications of Language (2019, background): “The free-text explanations are crucial to our formalism, as they can both increase trust in predictions made by the machine (Kulesza et al., 2012; Bussone et al., 2015; Nguyen et al., 2018) and encourage a poster’s empathy towards a targeted group, thereby combating biases (CohenAlmagor, 2014).”
Explanations Can Reduce Overreliance on AI Systems During Decision-Making (2022, background): “Mitigating effects of trust: In light of trust’s well-documented relation to reliance [4, 9, 10, 55, 59, 96, 97], we aim to neutralize any effects of trust in our task such that any differences in overreliance are attributable to our cost-benefit manipulations and not to individual differences in tr”
A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems (2018, background): “2015 [26] ◆ ◆ ◆”
F1 Is Not Enough! Models and Evaluation towards User-Centered Explainable Question Answering (2020, background): “, 2019), several performance variables indicating how well they judged the correctness of the model (fraction of correct ratings, false positive ratio (FP), false negative ratio (FN), true positive ratio (TP), true negative ratio (TN), precision (P), recall (R) and F1 values), agreement (fraction of”
An Evaluation of the Human-Interpretability of Explanation (2019, background): “, “this result makes sense to the human reader”) [Caruana et al., 2015, Lei et al., 2016, Ribeiro et al., 2016]. Human Factors in Explanation In parallel, the literature on explanation in psychology also offers several general insights into the design of interpretable AI systems. For example, humans”
Designing Theory-Driven User-Centric Explainable AI (2019, background): “The growing field of eXplainable AI (XAI) has produced algorithms to generate explanations as short rules [60, 63, 87], attribution or influence scores [15, 24, 74, 86], prototype examples, partial dependence plots [16, 56], etc.”
Explanations in Autonomous Driving: A Survey (2021, background): “influence scores [75] (such as partial dependence plots [76]) without sufficient justification for the explanation choices made are assumed to be based on unvalidated guidelines.”
To Trust or to Think (2021, methodology): “Consistent with prior research [7, 9, 26, 41], our results demonstrate that people aided by simple explainable AI interfaces performed better overall than people who completed the task without any AI support.”
A systematic review and taxonomy of explanations in decision support and recommender systems (2017, background): “(2015), Bussone et al. (2015), Muhammad et al. (2016), Wang et al. (2016b), Murphy and Phillips (1991), Suermondt and Cooper (1993), Swinney (1995), Ye (1995), Ramberg (1996), Nakatsu and Benbasat (2003), Gönül et al.”
Verbalization: Narration of Autonomous Robot Experience (2016, background): “The studies find that explanations increase trust of machine learning applications [Bussone et al., 2015] as well as improve users’ mental models.”
How do Humans Understand Explanations from Machine Learning Systems? An Evaluation of the Human-Interpretability of Explanation (2018, background): “Bussone et al. [2015] found that overly detailed explanations from clinical decision support systems enhance trust but also create over-reliance; short or absent explanations prevent over-reliance but decrease trust.”
Evaluating Explanation Without Ground Truth in Interpretable Machine Learning (2019, background): “…human-computer interaction (HCI) researchers actively explore this area ( 10,44–46 ), and propose several metrics for human evaluation on general explanations from IML, such as mental mode ( 15, 47–50 ), human-machine performance ( 51–55 ), user satisfaction ( 9, 56–58 ) and user trust ( 59–61 ).”
Trends and Trajectories for Explainable, Accountable and Intelligible Systems: An HCI Research Agenda (2018, background): “Domains include recommenders [33], decision support [62], medical [19, 99], business processes [88, 98], e-commerce [164, 186], social networks [63, 161], smart sensors [112, 113], smart homes [2, 30, 183], games [134, 182], and public displays [179].”
Current Challenges and Future Opportunities for XAI in Machine Learning-Based Clinical Decision Support Systems: A Systematic Review (2021, background): “[75] found that clinicians wanted better explanations from the CDSS to help them interpret the system’s confidence, to verify that the clinical disorder fit the CDSS suggestion, to better understand the reasoning chain of the system, and to make different diagnoses in order to help them make an asse”