Peeking Inside the Black Box: Visualizing Statistical Learning With Plots of Individual Conditional Expectation

By A. Goldstein, Adam Kapelner, Justin Bleich, E. Pitkin. 2013

1,731 citations recorded by the source corpus.

This article presents individual conditional expectation (ICE) plots, a tool for visualizing the model estimated by any supervised learning algorithm. Classical partial dependence plots (PDPs) help visualize the average partial relationship between the predicted response and one or more features. In the presence of substantial interaction effects, the partial response relationship can be heterogeneous. Thus, an average curve, such as the PDP, can obfuscate the complexity of the modeled relationship. Accordingly, ICE plots refine the PDP by graphing the functional relationship between the predicted response and the feature for individual observations. Specifically, ICE plots highlight the variation in the fitted values across the range of a covariate, suggesting where and to what extent heterogeneities might exist. In addition to providing a plotting suite for exploratory analysis, we include a visual test for additive structure in the data-generating model. Through simulated examples and real datasets, we demonstrate how ICE plots can shed light on estimated models in ways PDPs cannot. Procedures outlined are available in the R package ICEbox.

Subfields: Artificial Intelligence, Experimental and Cognitive Psychology, Statistics and Probability, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Experimental and Cognitive Psychology

In Statistics and Probability

In Safety, Robustness & Interpretability

Citations in the corpus

Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI (2019, positive): “Finally, [238] presents ICE (Individual Conditional Expectation) plots as a tool for visualizing the model estimated by any supervised learning algorithm.”
On the Art and Science of Machine Learning Explanations (2018, background): “Partial dependence and ICE can be combined in the same plot to identify interactions modeled by g and to create a holistic portrait of the predictions of a complex model for some X j [12].”
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI) (2018, methodology): “[112] introduced ICE techniques and proved it advantage over PDP.”
Visualizing the Feature Importance for Black Box Models (2018, methodology): “ty and interpretability [14]. Many researchers have therefore developed and implemented several model-agnostic interpretability tools, which quantify or visualize feature eects or feature importance [9, 11, 17]. In our context, the terms feature eect, feature contribution and feature attribution des”
Explanations of model predictions with live and breakDown packages (2018, methodology): “This method has many extensions such as for example Individual Conditional Expectations (Goldstein et al. (2015)).”
A Survey of Methods for Explaining Black Box Models (2018, methodology): “provide in Reference [35] a technique f that extends classical PDP named Individual Conditional Expectation (ICE) to visualize the model approximated by a black box b that helps in visualizing the average partial relationship between the outcome and some features.”
Forest Floor Visualizations of Random Forests (2016, background): “The iceBOX package displays multiple projection lines to search for masked interactions and is a good alternative to the forest floor approach [7].”
iml: An R package for Interpretable Machine Learning (2018, methodology): “• Individual conditional expectation (Goldstein et al. 2015): Visualizing the learned relationship between features and predictions for individual instances of the data.”
Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence (2023, neutral): “Goldstein et al. supported the same [133].”
pyCeterisParibus: explaining Machine Learning models with Ceteris Paribus Profiles in Python (2019, methodology): “The idea is an extension of PDP (Partial Dependency Plots) (Friedman, 2001) and ICE (Individual Conditional Expectations) plots (Goldstein, Kapelner, Bleich, & Pitkin, 2015).”
A Survey on the Explainability of Supervised Machine Learning (2020, methodology): “ underlying idea of partial dependence aims at showing how an individual feature aects the prediction of the global model. An individual feature’s relationship with the target is visualized in PDPs (Goldstein et al., 2015). Prospector uses an extension of PDP to visualize how features aect a predict”
Interpretable Machine Learning - A Brief History, State-of-the-Art and Challenges (2020, background): “Popular feature eﬀect plots are partial dependence plots [32], individual conditional expectation curves [39], accumulated local eﬀect plots [7], and the functional ANOVA [50].”
Human-Centered Explainable AI (XAI): From Algorithms to User Experiences (2021, methodology): “What if • Show how the prediction changes corresponding to the inquired change of input PDP [47], ALE [9], ICE [42]”
On the Interpretability of Artificial Intelligence in Radiology: Challenges and Opportunities. (2020, background): “Basic approaches to visualize the importance of input features to a model’s output include partial dependence plots (PDPs) and individual conditional expectation (ICE) plots (16), which are both methods for black-box models that aim to show the dependency between a model’s features and predictions.”
Explainability fact sheets: a framework for systematic assessment of explainable approaches (2019, background): “The second one comprises all sort of plots that can be used to help the user comprehend a predictive system’s behaviour, e.g., Individual Conditional Expectation [12] or Partial Dependence [10] plots.”
Are Explanations Helpful? A Comparative Study of the Effects of Explanations in AI-Assisted Decision-Making (2021, methodology): “Typical methods include ask people to rank the input data features based on their inuence to overall predictions [18, 33], to indicate the direction of change in the model’s prediction when a feature’s value is altered [16, 18, 29], to simulate the model’s predictions [16, 24, 40, 47, 59], to answer”