The Price of Interpretability

By Dimitris J. Bertsimas, Arthur Delarue, Patrick Jaillet, Sébastien Martin. 2019

39 citations recorded by the source corpus.

When quantitative models are used to support decision-making on complex and important topics, understanding a model's ``reasoning'' can increase trust in its predictions, expose hidden biases, or reduce vulnerability to adversarial attacks. However, the concept of interpretability remains loosely defined and application-specific. In this paper, we introduce a mathematical framework in which machine learning models are constructed in a sequence of interpretable steps. We show that for a variety of models, a natural choice of interpretable steps recovers standard interpretability proxies (e.g., sparsity in linear models). We then generalize these proxies to yield a parametrized family of consistent measures of model interpretability. This formal definition allows us to quantify the ``price'' of interpretability, i.e., the tradeoff with predictive accuracy. We demonstrate practical algorithms to apply our framework on real and synthetic datasets.

Subfields: Artificial Intelligence, Safety Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Maryam Ashoori
2. Kun Qian
3. Ban Kawas
4. Justin D. Weisz
5. Marina Danilevsky
6. Yannis Katsis
7. Solon Barocas
8. Manish Raghavan

In Safety Research

1. Manish Raghavan
2. Maryam Ashoori
3. Emily Black
4. Justin D. Weisz
5. Solon Barocas
6. Adrian Weller

Citations in the corpus

• In AI We Trust? Factors That Influence Trustworthiness of AI-infused Decision-Making Processes (2019, background): “Some kinds of AI models are inter-pretable, such as decision trees or rule-based scoring systems [6, 36].”
• A Survey of the State of Explainable AI for Natural Language Processing (2020, background): “In XAI, some works have claimed that explainability may come at the price of losing predictive quality (Bertsimas et al., 2019), while other have claimed the opposite (Garneau et al., 2018; Liang et al., 2016).”
• Model Multiplicity: Opportunities, Concerns, and Solutions (2022, background): “The existence of model multiplicity presents exciting opportunities because it offers model makers the flexibility to prioritize, and optimize for, desirable properties at no cost to accuracy, contrary to some conventionalwisdom [7, 16, 28, 42, 57, 60, 62].”