Interpretable Machine Learning - A Brief History, State-of-the-Art and Challenges

By Christoph Molnar, Giuseppe Casalicchio, Bernd Bischl. 2020

546 citations recorded by the source corpus.

We present a brief history of the field of interpretable machine learning (IML), give an overview of state-of-the-art interpretation methods, and discuss challenges. Research in IML has boomed in recent years. As young as the field is, it has over 200 years old roots in regression modeling and rule-based machine learning, starting in the 1960s. Recently, many new IML methods have been proposed, many of them model-agnostic, but also interpretation techniques specific to deep learning and tree-based ensembles. IML methods either directly analyze model components, study sensitivity to input perturbations, or analyze local or global surrogate approximations of the ML model. The field approaches a state of readiness and stability, with many methods not only proposed in research, but also implemented in open-source software. But many important challenges remain for IML, such as dealing with dependent features, causal interpretation, and uncertainty estimation, which need to be resolved for its successful application to scientific problems. A further challenge is a missing rigorous definition of interpretability, which is accepted by the community. To address the challenges and advance the field, we urge to recall our roots of interpretable, data-driven modeling in statistics and (rule-based) ML, but also to consider other areas such as sensitivity analysis, causal inference, and the social sciences.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Mathematical optimization in classification and regression trees (2021, background): “This is an example of local explainability , (Molnar et al. 2020), in which the aim is to identify a small set of features and their corresponding value to make the prediction change.”
Trust in AI: progress, challenges, and future directions (2024, unknown): “Despite these challenges, it is hoped that interpretability will soon reach a state of readiness(Molnar et al., 2020).”
Explainable AI (XAI): A Systematic Meta-Survey of Current Challenges and Future Opportunities (2021, methodology): “However, it seems that there are two main evaluation metrics groups: objective and human-centered evaluations [32].”
A Survey on Neural Network Interpretability (2020, background): “, [18], [49], [58], [64]), where global interpretability means being able to understand the overall decision logic of a model and local interpretability focuses on the explanations of”
Interactive Model Cards: A Human-Centered Approach to Model Documentation (2022, background): “In an effort to mitigate the risks for harm in AI/ML systems, researchers and developers have proposed strategies to make systems more explainable, transparent, and contestable (e.g., [3, 24, 34, 41, 45, 47, 74]).”
Evaluating the Quality of Machine Learning Explanations: A Survey on Methods and Metrics (2021, methodology): “Model size [14,46] Runtime operation counts [81] Interaction strength [23,46] Main effect complexity [46] Level of (dis)agreement [82] Attribution-based explanations Monotonicity [80] Non-sensitivity [80,83], Sensitivity [84,85] Effective complexity [80] Remove and retrain [86] Recall of important…”
Current Challenges and Future Opportunities for XAI in Machine Learning-Based Clinical Decision Support Systems: A Systematic Review (2021, methodology): “There are two main ways of evaluating post-hoc methods: mathematically quantiﬁable metrics and human-centered evaluations [67].”