Please Stop Permuting Features: An Explanation and Alternatives

By Giles Hooker, Lucas K. Mentch. 2019

129 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Explainable machine learning in deployment (2019, background): “how the model behaves upon an infinitesimal perturbation (not necessarily a feasible one [30]), but does not “explain” if the model captures the underlying causal mechanism in the data.”
Machine learning and deep learning—A review for ecologists (2022, methodology): “Although there are still many questions about the reliability of these methods, especially under collinearity of features (Hooker & Mentch, 2019; Yu et al., 2021 , but see Apley & Zhu, 2020 ), xAI is becoming an indispensable tool for working with ML models, and presumably there will be specialized…”
Interpretable Machine Learning - A Brief History, State-of-the-Art and Challenges (2020, methodology): “Correlated features in random forests are preferred and attributed a higher importance [110,51].”
Explaining by Removing: A Unified Framework for Model Explanation (2020, background): “The choice of how to remove features has been considered by many works (Lundberg and Lee, 2017; Chang et al., 2018; Janzing et al., 2019; Sundararajan and Najmi, 2019; Merrick and Taly, 2019; Aas et al., 2019; Hooker and Mentch, 2019; Agarwal and Nguyen, 2019; Frye et al., 2020).”
Understanding Global Feature Contributions With Additive Importance Measures (2020, methodology): “The canonical method for this is a feature ablation study where, in addition to a model f trained on all features, separate models f 1 , f 2 , . . . , f d are trained with individual features excluded [2, 15, 20].”
Problems with Shapley-value-based explanations as feature importance measures (2020, background): “This “out-of-distribution” phenomenon has been explored recently by Hooker & Mentch (2019), who show why “permutation-based” methods to evaluate feature importance can be highly misleading: when values are substituted into feature set S̄ that are unlikely or impossible when conditioned on feature…”
Epistemic values in feature importance methods: Lessons from feminist epistemology (2021, background): “veridical models of reality by deleting extraneous information, and this is precisely the reason that permutation feature importance, his approximation of the definition he intuited, has been shown to be problematic [37].”