A Human-Grounded Evaluation Benchmark for Local Explanations of Machine Learning

By Sina Mohseni, Eric D. Ragan. 2018

63 citations recorded by the source corpus.

In order for people to be able to trust and take advantage of the results of advanced machine learning and artificial intelligence solutions for real decision making, people need to be able to understand the machine rationale for given output. Research in explain artificial intelligence (XAI) addresses the aim, but there is a need for evaluation of human relevance and understandability of explanations. Our work contributes a novel methodology for evaluating the quality or human interpretability of explanations for machine learning models. We present an evaluation benchmark for instance explanations from text and image classifiers. The explanation meta-data in this benchmark is generated from user annotations of image and text samples. We describe the benchmark and demonstrate its utility by a quantitative evaluation on explanations generated from a recent machine learning algorithm. This research demonstrates how human-grounded evaluation could be used as a measure to qualify local machine-learning explanations.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Information Systems and Management

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Information Systems and Management

Citations in the corpus

Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI) (2018, methodology): “Based on Doshi-Velez’s evaluation classification, Mohseni and Ragan [148] presented a human-grounded evaluation benchmark for evaluating instance explanations of images and textual data.”
Aggregating explainability methods for neural networks stabilizes explanations (2019, methodology): “We use the benchmark introduced in (Mohseni & Ragan, 2018) to test the alignment between human evaluation and explanation methods.”
Human-grounded Evaluations of Explanation Methods for Text Classification (2019, background): “For human-grounded evaluation, Mohseni and Ragan (2018) proposed a benchmark which contains a list of relevant words for the actual class of each input text, identiﬁed by human experts.”
A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems (2018, methodology): “Mohseni et al. [149] proposed a multi-layer Human-Attention baseline for feature-level evaluation of machine learning explanations.”
Interpretable Machine Learning - A Brief History, State-of-the-Art and Challenges (2020, background): “Research-wise, the ﬁeld is maturing in terms of methods surveys [75,41,120,96,1,6,23,15], further consolidation of terms and knowledge [42,22,82,97,88,17], and work about deﬁning interpretability or evaluation of IML methods [74,73,95,49].”
Trustworthy AI: From Principles to Practices (2021, unknown): “Development Functional Testing [164, 235, 381] Performance Benchmarking [40, 56, 88, 93, 123, 127, 153, 238, 248, 307, 327]”
Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey (2020, background): “• Human-grounded Evaluation Benchmark: In [110], Moshseni et al.”
Explainable, trustworthy, and ethical machine learning for healthcare: A survey (2021, methodology): “[172] conducted a human-grounded study and evaluated the performance of the LIME algorithm by comparing the explanation produced by LIME with the weighted explanations generated by 10 humans experts.”
Aggregating explanation methods for stable and robust explainability. (2019, methodology): “While human annotation is expensive, there exists a benchmark for human evaluation introduced in (Mohseni & Ragan, 2018).”
Evaluating Explanation Methods for Neural Machine Translation (2020, background): “Mohseni and Ragan (2018) propose a paradigm to evaluate explanation methods for document classiﬁcation that involves human judgment for evaluation.”