Interpretable Deep Neural Networks for Single-Trial EEG Classification

By Irene Sturm, Sebastian Bach, Wojciech Samek, K. Müller. 2016

388 citations recorded by the source corpus.

BACKGROUND In cognitive neuroscience the potential of deep neural networks (DNNs) for solving complex classification tasks is yet to be fully exploited. The most limiting factor is that DNNs as notorious 'black boxes' do not provide insight into neurophysiological phenomena underlying a decision. Layer-wise relevance propagation (LRP) has been introduced as a novel method to explain individual network decisions. NEW METHOD We propose the application of DNNs with LRP for the first time for EEG data analysis. Through LRP the single-trial DNN decisions are transformed into heatmaps indicating each data point's relevance for the outcome of the decision. RESULTS DNN achieves classification accuracies comparable to those of CSP-LDA. In subjects with low performance subject-to-subject transfer of trained DNNs can improve the results. The single-trial LRP heatmaps reveal neurophysiologically plausible patterns, resembling CSP-derived scalp maps. Critically, while CSP patterns represent class-wise aggregated information, LRP heatmaps pinpoint neural patterns to single time points in single trials. COMPARISON WITH EXISTING METHOD(S) We compare the classification performance of DNNs to that of linear CSP-LDA on two data sets related to motor-imaginary BCI. CONCLUSION We have demonstrated that DNN is a powerful non-linear tool for EEG analysis. With LRP a new quality of high-resolution assessment of neural activity can be reached. LRP is a potential remedy for the lack of interpretability of DNNs that has limited their utility in neuroscientific applications. The extreme specificity of the LRP-derived heatmaps opens up new avenues for investigating neural activity underlying complex perception or decision-related processes.

Subfields: Cognitive Neuroscience, Safety, Robustness & Interpretability, Neuroscience

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Safety, Robustness & Interpretability

In Neuroscience

Citations in the corpus

A Survey of Methods for Explaining Black Box Models (2018, methodology): “The only exceptions are [39, 113] that using EEG data provide heatmaps with data point’s relevance for the decision’s outcome.”
Methods for interpreting and understanding deep neural networks (2017, methodology): “de (Klaus-Robert Müller) sights from complex physical, chemical, or biological systems [20, 21, 49, 43, 54].”
Explaining nonlinear classification decisions with deep Taylor decomposition (2015, methodology): “2See [42, 43] for the application of decomposition techniques to text and EEG data.”
Unmasking Clever Hans predictors and assessing what machines really learn (2019, methodology): “A comparison of both approaches is given in Supplementary Note 6 and Supplementary Figures 21 and 22.”
A cloud detection algorithm for satellite imagery based on deep learning (2019, methodology): “, 2018), where methods for interpretability has been used to gain insights in physical, chemical, and biological systems (Alipanahi et al., 2015; Schütt et al., 2017; Sturm et al., 2016).”
Artificial Intelligence in Surgery: Promises and Perils (2018, background): “54,55 However, many of these efforts remain in their infancy, and surgeon input early in the design of AI algorithms may be helpful improving accountability and interpretability of big data analyses.”
On Interpretability of Artificial Neural Networks: A Survey (2020, methodology): “I. Sturm et al. [166] applied a deep network with LRP [11] for the single-trial EEG [22] classification.”
Interpreting and Explaining Deep Neural Networks for Classification of Audio Signals (2018, background): “Latter ones were originally successfully applied to image classifiers and have more recently also been transferred to other domains such as natural language processing [8, 9], EEG analysis [10] or physics [11].”
Deep learning-based electroencephalography analysis: a systematic review (2019, background): “More recently, studies such as [75, 183] achieved promising results without the need to extract features.”
Analyzing Neuroimaging Data Through Recurrent Deep Learning Models (2018, neutral): “Recently, researchers have already demonstrated the usefulness of interpretable DLmethods for the analysis of EEG data (Sturm et al., 2016).”