Evaluating and Aggregating Feature-based Model Explanations

By Umang Bhatt, Adrian Weller, José M. F. Moura. 2020

313 citations recorded by the source corpus.

A feature-based model explanation denotes how much each input feature contributes to a model's output for a given data point. As the number of proposed explanation functions grows, we lack quantitative evaluation criteria to help practitioners know when to use which explanation function. This paper proposes quantitative evaluation criteria for feature-based explanations: low sensitivity, high faithfulness, and low complexity. We devise a framework for aggregating explanation functions. We develop a procedure for learning an aggregate explanation function with lower complexity and then derive a new aggregate Shapley value explanation function that minimizes sensitivity.

Subfields: Artificial Intelligence, Information Systems and Management

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems and Management

Adrian Weller

Citations in the corpus

Getting a CLUE: A Method for Explaining Uncertainty Estimates (2020, background): “Using the same conﬁguration, we generate LIME and SHAP explanations for some MNIST digits to which our BNN assigns predictive entropy above our rejection threshold.”
To Trust or to Think (2021, background): “Hence, in addition to tuning explanation attributes such as soundness and completeness [39], or faithfulness, sensitivity, and complexity [5], explainable AI researchers should ensure that people will exert effort to attend to those explanations.”
From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI (2022, background): “Explanations should be sparse, short and not redundant to avoid presenting an explanation that is too big to understand [26, 57, 166, 233, 237, 314].”