A Roadmap for a Rigorous Science of Interpretability

By Finale Doshi-Velez, Been Kim. 2017

158 citations recorded by the source corpus.

Subfields: Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Safety, Robustness & Interpretability

Citations in the corpus

Explanation in Human-AI Systems: A Literature Meta-Review, Synopsis of Key Ideas and Publications, and Bibliography for Explainable AI (2019, background): “…2012, 2011, 2013, 2015; Nataksu, 2004; Ribiero, et al., 2016a,b; Sheh and Monteath, 2008; Treu, 1998; Wickens, 1994), and the challenges for the evaluation of explanation systems (Doshi-Velez and Kim, 2017ab; Gilpin et al., 2018; Feldmann, 2018; Miller, 2017; Miller, Howe, and Sonenberg, 2017).”
Interpretability via Model Extraction (2017, background): “, how model predictions are affected by changing the prejudiced feature [12].”
Computing Optimal Decision Sets with SAT (2020, background): “Hence the rising interest in explainable artificial intelligence (XAI) [4, 14, 16, 18, 22, 23, 26, 29, 30, 36–38, 42, 43, 48, 49, 52], including research programs [3, 24] and legislation [17, 21].”
DeltaGrad: Rapid retraining of machine learning models (2020, methodology): “For example, repeated retraining by removing different sub-sets of training data each time is essential in many existing data systems (Doshi-Velez & Kim, 2017; Krishnan & Wu, 2017) to understand the effect of those removed data over the model behavior.”
A causal framework for explaining the predictions of black-box sequence-to-sequence models (2017, background): “A good explanation would ideally also decompose into cognitive chunks (Doshi-Velez and Kim, 2017): basic units of explanation which are a priori bounded in size.”
Interpreting Blackbox Models via Model Extraction (2017, background): “This approach requires the ability to understand the model’s reasoning process, i.e., how model predictions are affected by changing the prejudiced feature [21].”
AI Safety Gridworlds (2017, background): “Nevertheless, we needed to omit some safety problems such as interpretability (Doshi-Velez and Kim, 2017), multi-agent problems (Chmait et al.”
How do Humans Understand Explanations from Machine Learning Systems? An Evaluation of the Human-Interpretability of Explanation (2018, background): “Hayete and Bienkowska [2004] found a preference for non-oblique splits in decision trees (see Freitas [2014] for more detailed survey).”
Abduction-Based Explanations for Machine Learning Models (2018, background): “The importance of computing explanations is further underscored by a number of recent works (Goodman and Flaxman 2017; Doshi-Velez and Kim 2017; Monroe 2018; Darwiche 2018; Lip-ton 2018), by recent regulations (EU Data Protection Regulation 2016), ongoing research programs (DARPA 2016), but also by…”
Explanation and Justification in Machine Learning : A Survey Or (2017, background): “More recently, [Doshi-Velez and Kim, 2017] considered how to evaluate human interpretability of machine learning models.”
Toward Interpretable Machine Learning: Transparent Deep Neural Networks and Beyond (2020, background): “Application-driven evaluations account for those factors, however, they are also hard to implement in practice [34].”
Simplicity Creates Inequity: Implications for Fairness, Stereotypes, and Interpretability (2018, unknown): “hat produces pure numerical predictions with no accompanying insight? And similarly, can an applicant derive any understanding about the basis for the algorithm’s prediction in their particular case? [11, 28, 38] Fairness and interpretability are clearly distinct issues, but it is natural to suspect”