Evaluating the Quality of Machine Learning Explanations: A Survey on Methods and Metrics

By Jianlong Zhou, Amir H. Gandomi, Fang Chen, Andreas Holzinger. Electronics, 2021

622 citations recorded by the source corpus.

The most successful Machine Learning (ML) systems remain complex black boxes to end-users, and even experts are often unable to understand the rationale behind their decisions. The lack of transparency of such systems can have severe consequences or poor uses of limited valuable resources in medical diagnosis, financial decision-making, and in other high-stake domains. Therefore, the issue of ML explanation has experienced a surge in interest from the research community to application domains. While numerous explanation methods have been explored, there is a need for evaluations to quantify the quality of explanation methods to determine whether and to what extent the offered explainability achieves the defined objective, and compare available explanation methods and suggest the best explanation from the comparison for a specific task. This survey paper presents a comprehensive overview of methods proposed in the current literature for the evaluation of ML explanations. We identify properties of explainability from the review of definitions of explainability. The identified properties of explainability are used as objectives that evaluation metrics should achieve. The survey found that the quantitative metrics for both model-based and example-based explanations are primarily used to evaluate the parsimony/simplicity of interpretability, while the quantitative metrics for attribution-based explanations are primarily used to evaluate the soundness of fidelity of explainability. The survey also demonstrated that subjective measures, such as trust and confidence, have been embraced as the focal point for the human-centered evaluation of explainable systems. The paper concludes that the evaluation of ML explanations is a multidisciplinary research topic. It is also not possible to define an implementation of evaluation metrics, which can be applied to all explanation methods.

Subfields: Artificial Intelligence, Health Informatics, Information Systems and Management

View Paper: https://www.semanticscholar.org/paper/49774cf91b00381501c960fc976e982c0eb51b68

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Health Informatics

In Information Systems and Management

Citations in the corpus

Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directions (2023, background): “Furthermore, XAI has evolved into an exceedingly multidisciplinary, interdisciplinary, and transdisciplinary field.”
A comprehensive taxonomy for explainable artificial intelligence: a systematic survey of surveys on methods and concepts (2021, methodology): “A full focus on metrics for evaluating XAI methods is set in the recent work [151] by Zhou et al.”
A Review of Taxonomies of Explainable Artificial Intelligence (XAI) Methods (2022, background): “As a research discipline, XAI has grown exponentially in recent years, with annual publications easily exceeding several hundreds [10, 75].”
From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI (2022, background): “[314] that presents a concise overview and discussion regarding evaluating XAI without a structured literature review, and the recent work of Vilone and Longo [263] that present a list of concepts related to explainability and discuss evaluation methods in combination with explainable AI methods.”
Current Challenges and Future Opportunities for XAI in Machine Learning-Based Clinical Decision Support Systems: A Systematic Review (2021, background): “A summary of many quantitative metrics for the evaluation of explainability properties (i.e., clarity, broadness, parsimony, completeness, and soundness) for different explanation types, is presented in the work of Zhou et al. [141].”