Visualizing the effects of predictor variables in black box supervised learning models

By Daniel W. Apley, Jingyu Zhu. 2016

1,454 citations recorded by the source corpus.

In many supervised learning applications, understanding and visualizing the effects of the predictor variables on the predicted response is of paramount importance. A shortcoming of black box supervised learning models (e.g. complex trees, neural networks, boosted trees, random forests, nearest neighbours, local kernel‐weighted methods and support vector regression) in this regard is their lack of interpretability or transparency. Partial dependence plots, which are the most popular approach for visualizing the effects of the predictors with black box supervised learning models, can produce erroneous results if the predictors are strongly correlated, because they require extrapolation of the response at predictor values that are far outside the multivariate envelope of the training data. As an alternative to partial dependence plots, we present a new visualization approach that we term accumulated local effects plots, which do not require this unreliable extrapolation with correlated predictors. Moreover, accumulated local effects plots are far less computationally expensive than partial dependence plots. We also provide an R package ALEPlot as supplementary material to implement our proposed method.

Subfields: Artificial Intelligence, Nature and Landscape Conservation, Statistics and Probability, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Nature and Landscape Conservation

In Statistics and Probability

In Safety, Robustness & Interpretability

Citations in the corpus

Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI (2019, background): “Informativeness [5, 44, 21, 25, 26, 45, 30, 32, 34, 35, 37, 38, 41, 46, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 63, 64, 65, 66, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 86, 87, 88, 89, 59, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 1”
Explanations of model predictions with live and breakDown packages (2018, methodology): “On the other hand, ALE plots (Apley (2016)) were proposed as a superior tool for handling strongly correlated predictors by describing conditional distribution of predicted values.”
Questioning the AI: Informing Design Practices for Explainable AI User Experiences (2020, methodology): “Provide example(s) similar to the instance and with the same record as the prediction [13, 48, 50] Why , How to still be this”
TOWARDS HIERARCHICAL IMPORTANCE ATTRIBU- (2019, background): “, 2009), Accumulated Local Effects (Apley, 2016).”
Statistical stability indices for LIME: Obtaining reliable explanations for machine learning models (2020, methodology): “Based on variable exclusion, the predictive power of the ML models has been decomposed into single variables contribution in PDP (Friedman, 2001), ICE (Goldstein, Kapelner, Bleich, & Pitkin, 2015) and ALE (Apley & Zhu, 2016) plots, based on diﬀerent assumptions about the ML model.”
A historical perspective of explainable Artificial Intelligence (2020, methodology): “The example shows that when features of a machine learning model are correlated, PDPs are not very accurate and cannot be trusted (Apley & Zhu, 2016) FIGURE 4 Local explanation extracted through LIME in the Boston dataset (Harrison & Rubinfeld, 1978).”
The role of explainability in creating trustworthy artificial intelligence for health care: a comprehensive survey of the terminology, design choices, and evaluation strategies (2020, neutral): “12 Two examples of visual global attribution methods are Partial Dependency Plots (PDPs) [44] and Accumulated Local Effects (ALE) plots [43], both show the average effect of input features to the output.”
Locally Interpretable Models and Effects based on Supervised Partitioning (LIME-SUP) (2018, methodology): “…for diagnosing interactions (Friedman & Popescu, 2005), related ones such as ICE (Goldstein, Kapelner, Bleich, & Pitkin, 2013) and ALE plots (Apley, 2016), derivative-based analyses (Kucherenko, 2010; Sobol & Kucherenko, 2009), Sobol indices (Iooss & Lemaitre, 2014), Shapley analyses…”
Counterfactual Explanations for Machine Learning: A Review (2020, neutral): “Dependency plots are a popular kind of visual explanation, e.g., Partial Dependence Plots [51], Accumulated Local Effects Plot [14], Individual Conditional Expectation [53].”
Machine learning and deep learning—A review for ecologists (2022, positive): “…many questions about the reliability of these methods, especially under collinearity of features (Hooker & Mentch, 2019 ; Yu et al., 2021 , but see Apley & Zhu, 2020 ), xAI is becoming an indispensable tool for working with ML models, and presumably there will be specialized xAI methods for…”
Explainable AI: A Review of Machine Learning Interpretability Methods (2020, methodology): “Another method that is closely-related to PDPs is the Accumulated Local Effect (ALE) plots [61].”
Counterfactual Explanations and Algorithmic Recourses for Machine Learning: A Review (2020, neutral): “Dependency plots are a popular kind of visual explanation, e.g., Partial Dependence Plots [119], Accumulated Local Effects Plot [16], and Individual Conditional Expectation [131].”
Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence (2023, background): “Zhu [137], ALE plots are always the forerunner when it comes to visualising features that are correlated.”
Human-Centered Explainable AI (XAI): From Algorithms to User Experiences (2021, background): “What if • Show how the prediction changes corresponding to the inquired change of input PDP [47], ALE [9], ICE [42]”
Machine Learning Interpretability: A Survey on Methods and Metrics (2019, background): “Partial Dependence Plot [142] Global Feature summary Individual Condition Expectation [143] Global/Local Feature summary Accumulated Local Effects Plot [144] Global Feature summary Feature Interaction [145] Global Feature summary Feature Importance [146] Global/Local Feature summary Local Surrogate ”
Interpretable Machine Learning - A Brief History, State-of-the-Art and Challenges (2020, methodology): “We have a better understanding of weaknesses of IML methods in general [75,79], but also specifically for methods such as permutation feature importance [51,110,7,111], Shapley values [57,113], counterfactual explanations [63], partial dependence plots [51,50,7] and saliency maps [2].”