Explaining Therapy Predictions with Layer-Wise Relevance Propagation in Neural Networks

By Yinchong Yang, Volker Tresp, Marius Wunderle, Peter A. Fasching. 2018

79 citations recorded by the source corpus.

In typical data analysis projects in biology and healthcare, simpler predictive models, such as regressions and decision trees, enjoy more popularity than more complex and expressive ones, such as neural networks. One reason for this is that the functioning of simpler models is easier to explain, which greatly increases user acceptance. A neural network, on the contrary, is often regarded as a black box model, because its very strength in modeling complex interactions also makes its operation almost impossible to explain. Still, neural networks remain very interesting tools, since they have demonstrated promising performance in a variety of predictive tasks, such as medical image classification and segmentation, as well as clinical event prediction, i.e., in the modeling of therapy decisions and survival time. In this work, we attempt to improve the explainability of neural networks applied in healthcare. We propose to apply the Layer-wise Relevance Propagation algorithm to explain clinical decisions proposed by deep modern neural networks. This algorithm is able to highlight the features that lead to the probabilistic prediction of therapy decisions for each individual patient. We evaluate the feature-oriented explanations generated by the algorithm with clinical experts. We show that the features, which are identified by the algorithm to be relevant, largely agree with clinical knowledge and guidelines. We believe that being able to explain machine learning based decisions greatly improves transparency and acceptance of neural network models applied in the clinical domain.

Subfields: Artificial Intelligence, Health Informatics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Health Informatics

Citations in the corpus

Explaining and Interpreting LSTMs (2019, methodology): “Yet another work validated LSTM explanations via a toy classification task [77], however it practically treated the relevance as a binary variable.”
Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications (2021, background): “Yet, another work [196] applies LSTM predictors together with LRP for transparent therapy prediction on patients suffering from metastatic breast cancer.”
Toward Interpretable Machine Learning: Transparent Deep Neural Networks and Beyond (2020, background): “Yet another work [155] applies LSTM predictors together with LRP for transparent therapy prediction on patients suffering from metastatic breast cancer.”
Explainable AI: Interpreting, Explaining and Visualizing Deep Learning (2019, background): “They not only outperform humans in complex visual tasks [16,53] or strategic games [56,61,83], but also became an indispensable part of our every day lives, e.”
What Clinicians Want: Contextualizing Explainable Machine Learning for Clinical End Use (2019, background): “This area remains relatively unexplored in clinical ML explainability literature with a few exceptions (Xu et al., 2018; Choi et al., 2016; Yang et al., 2018) focused primarily on attention based and sensitivity analysis mechanisms in deep learning (Vaswani et al.”
Explainable, trustworthy, and ethical machine learning for healthcare: A survey (2021, methodology): “Yang et al. [107] proposed the use of LRP to select the features with high relevance for predicting the decision of therapy of patients.”