Improving the accuracy of medical diagnosis with causal machine learning

By Jonathan G. Richens, Ciarán M. Lee, Saurabh Johri. 2020

557 citations recorded by the source corpus.

Machine learning promises to revolutionize clinical decision making and diagnosis. In medical diagnosis a doctor aims to explain a patient’s symptoms by determining the diseases causing them. However, existing machine learning approaches to diagnosis are purely associative, identifying diseases that are strongly correlated with a patients symptoms. We show that this inability to disentangle correlation from causation can result in sub-optimal or dangerous diagnoses. To overcome this, we reformulate diagnosis as a counterfactual inference task and derive counterfactual diagnostic algorithms. We compare our counterfactual algorithms to the standard associative algorithm and 44 doctors using a test set of clinical vignettes. While the associative algorithm achieves an accuracy placing in the top 48% of doctors in our cohort, our counterfactual algorithm places in the top 25% of doctors, achieving expert clinical accuracy. Our results show that causal reasoning is a vital missing ingredient for applying machine learning to medical diagnosis. In medical diagnosis a doctor aims to explain a patient’s symptoms by determining the diseases causing them, while existing diagnostic algorithms are purely associative. Here, the authors reformulate diagnosis as a counterfactual inference task and derive new counterfactual diagnostic algorithms.

Subfields: Artificial Intelligence, Family Practice, Causal Inference & Discovery, Medical & Health AI

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Causal Inference & Discovery

In Medical & Health AI

Citations in the corpus

Toward Causal Representation Learning (2021, background): “While it may be useful to automate certain decisions, an understanding of causality may be necessary to recommend treatment options that are personalized and reliable [3], [6], [31], [164], [201], [224], [242], [273].”
Towards Causal Representation Learning (2021, background): “While it may be useful to automate certain decisions, an understanding of causality may be necessary to recommend treatment options that are personalized and reliable [201, 242, 224, 273, 6, 3, 30, 165].”
Deep Learning for Medical Anomaly Detection – A Survey (2020, background): “is often confused with association [125], [126], [127].”
Artificial Intelligence for Clinical Oncology (2021, background): “The concept of AI, formalized in the 1950s, was originally deﬁned as the ability of a machine to perform a task normally associated with human performance (Russell and Haller, 2003).”