Fairwashing Explanations with Off-Manifold Detergent

By Christopher J. Anders, Plamen Pasliev, Ann-Kathrin Dombrowski, K. Müller, Pan Kessel. 2020

107 citations recorded by the source corpus.

Explanation methods promise to make black-box classifiers more transparent. As a result, it is hoped that they can act as proof for a sensible, fair and trustworthy decision-making process of the algorithm and thereby increase its acceptance by the end-users. In this paper, we show both theoretically and experimentally that these hopes are presently unfounded. Specifically, we show that, for any classifier $g$, one can always construct another classifier $\tilde{g}$ which has the same behavior on the data (same train, validation, and test error) but has arbitrarily manipulated explanation maps. We derive this statement theoretically using differential geometry and demonstrate it experimentally for various explanation methods, architectures, and datasets. Motivated by our theoretical insights, we then propose a modification of existing explanation methods which makes them significantly more robust.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Attributing Fair Decisions with Attention Interventions (2021, background): “For instance, it has been previously shown that interpretability frameworks can be used as a means for fairwashing which is when malicious users generate fake explanations for their unfair decisions to justify them (Anders et al. 2020).”
Model Multiplicity: Opportunities, Concerns, and Solutions (2022, background): “The converse does not hold: two models with the same prediction on a given point may still exhibit variation in the process by which that outcome was reached [4, 11].”