The Doctor Just Won't Accept That!

By Zachary Chase Lipton. 2017

107 citations recorded by the source corpus.

Calls to arms to build interpretable models express a well-founded discomfort with machine learning. Should a software agent that does not even know what a loan is decide who qualifies for one? Indeed, we ought to be cautious about injecting machine learning (or anything else, for that matter) into applications where there may be a significant risk of causing social harm. However, claims that stakeholders "just won't accept that!" do not provide a sufficient foundation for a proposed field of study. For the field of interpretable machine learning to advance, we must ask the following questions: What precisely won't various stakeholders accept? What do they want? Are these desiderata reasonable? Are they feasible? In order to answer these questions, we'll have to give real-world problems and their respective stakeholders greater consideration.

Subfields: Artificial Intelligence, Safety Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety Research

Citations in the corpus

FairSight: Visual Analytics for Fairness in Decision Making (2019, background): “Feature selection often requires domain knowledge; determining how a feature is important and fair may differ across contexts and domains [27], and is also subjective to people’s perception on fairness [16].”
Reinforcement Learning in Healthcare: A Survey (2019, background): “These limitations therefore greatly hinder the successful adoption of RL policies for safety-critical applications such as in medical domains as clinicians are unlikely to try new treatments without rigorous validation for safety, correctness and robustness [133].”
A Unifying Review of Deep and Shallow Anomaly Detection (2020, background): “For researchers and practitioners alike [520], it is vital to understand the underlying reasons for how a specific AD model reaches a particular prediction.”
A Survey on the Explainability of Supervised Machine Learning (2020, background): “Lipton (2017) states that addressing the foundations of the problem by discovering what explainability really is will be crucial to see meaningful progress within this field.”
"The human body is a black box": supporting clinical decision-making with deep learning (2019, background): “ adopt, and use technologies are often an afterthought. Clinicians are rarely directly involved in the development of machine learning technologies [24] or discussions of explainability in healthcare [25]. Too often, the focus has been on the technical properties of the model, and in turn, the poten”
Explainable Deep Learning: A Field Guide for the Uninitiated (2020, background): “…explanation representation be detailed enough such that the doctor can understand the reasoning process behind the specific diagnosis and be confident about said diagnosis (Lipton, 2017), e.g., the patient needs this specific treatment because it identifies features of cancer at a particular stage.”