Benchmarking and survey of explanation methods for black box models

By Francesco Bodria, Fosca Giannotti, Riccardo Guidotti, Francesca Naretto, Dino Pedreschi, Salvatore Rinzivillo. Data mining and knowledge discovery, 2021

345 citations recorded by the source corpus.

The rise of sophisticated black-box machine learning models in Artificial Intelligence systems has prompted the need for explanation methods that reveal how these models work in an understandable way to users and decision makers. Unsurprisingly, the state-of-the-art exhibits currently a plethora of explainers providing many different types of explanations. With the aim of providing a compass for researchers and practitioners, this paper proposes a categorization of explanation methods from the perspective of the type of explanation they return, also considering the different input data formats. The paper accounts for the most representative explainers to date, also discussing similarities and discrepancies of returned explanations through their visual appearance. A companion website to the paper is provided as a continuous update to new explainers as they appear. Moreover, a subset of the most robust and widely adopted explainers, are benchmarked with respect to a repertoire of quantitative metrics.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/93a55b04045ff7fff78de473f5ff52cbcfb9a948

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Bo Li
2. Bowen Zhou
3. Longo Luca
4. Jinfeng Yi
5. Shuai Di
6. Jingen Liu
7. Jiquan Pei
8. Bo Liu

Citations in the corpus

• Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directions (2023, methodology): “In other words, despite the broad interest in the design of XAI methods [18,31,48–50], it is still unclear how to compare the results of different evaluations and establish a common understanding of how to evaluate explanations.”
• Trustworthy AI: From Principles to Practices (2021, methodology): “, (1) the explanation is fragile to perturbations [130], (2) the explanation is not always consistent with human interpretation [47], and (3) it is difficult to judge if the explanation is correct or faithful [250].”