Learning Representations from EEG with Deep Recurrent-Convolutional Neural Networks

By Pouya Bashivan, Irina Rish, Mohammed Yeasin, Noel C. F. Codella. International Conference on Learning Representations, 2015

765 citations recorded by the source corpus.

One of the challenges in modeling cognitive events from electroencephalogram (EEG) data is finding representations that are invariant to inter- and intra-subject differences, as well as to inherent noise associated with such data. Herein, we propose a novel approach for learning such representations from multi-channel EEG time-series, and demonstrate its advantages in the context of mental load classification task. First, we transform EEG activities into a sequence of topology-preserving multi-spectral images, as opposed to standard EEG analysis techniques that ignore such spatial information. Next, we train a deep recurrent-convolutional network inspired by state-of-the-art video classification to learn robust representations from the sequence of images. The proposed approach is designed to preserve the spatial, spectral, and temporal structure of EEG which leads to finding features that are less sensitive to variations and distortions within each dimension. Empirical evaluation on the cognitive load classification task demonstrated significant improvements in classification accuracy over current state-of-the-art approaches in this field.

Subfields: Artificial Intelligence, Cognitive Neuroscience

View Paper: https://www.semanticscholar.org/paper/2623847aeeac58f976a4f94d556bf8488b5d5b63

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

Citations in the corpus

Learning Temporal Information for Brain-Computer Interface Using Convolutional Neural Networks (2018, neutral): “[28] introduced a novel representation for EEG Signals by using an image”
An end-to-end deep learning approach to MI-EEG signal classification for BCIs (2018, methodology): “Bashivan et al. (2016) used CNNs to classify EEG signals through spectral topography maps generated from short-time Fourier transformed (STFT) recordings.”
Deep Learning Human Mind for Automated Visual Classification (2016, methodology): “During the experiment, 2,000 images (50 from each class) were shown in bursts for 0.5 seconds each.”
EEG-Based Spatio–Temporal Convolutional Neural Network for Driver Fatigue Evaluation (2019, positive): “In another work [30], by using fast Fourier transform”
A Deep Learning Architecture for Temporal Sleep Stage Classification Using Multivariate and Multimodal Time Series (2017, background): “Less classically and more recently various deep learning approaches have been proposed to learn from EEG data [21]–[24] and some of these contributions use a ﬁrst layer that boils down to a spatial ﬁlter [25]–[30].”
EEGNet: a compact convolutional neural network for EEG-based brain–computer interfaces (2016, methodology): “Recently, a recurrent convolutional network was proposed by [28] to predict mental workload.”
Data Augmentation for Deep-Learning-Based Electroencephalography. (2020, methodology): “16 Stu dy Dataset Task information Input formulat ion Deep learning strategy Noise Addition parameters Accura cy (witho ut DA) Accura cy (with DA) [50] University of Memphis Institutional review board 13 subjects 2670 trials 64 channels Mental workload”
Deep learning-based electroencephalography analysis: a systematic review (2019, methodology): “Several studies used backpropagation-based techniques to generate input maps that maximize activations of specific units [16, 158, 178, 207].”
EEG-Based Emotion Recognition Using Regularized Graph Neural Networks (2019, background): “In contrast, topology-aware classifiers such as CNNs [9], [36], [37], [38] and GNNs [12] consider the inter-channel topological relations and learn EEG representations for each channel by aggregating features from nearby channels using convolutional operations either in the Euclidean space or in the”
Deep learning for electroencephalogram (EEG) classification tasks: a review (2019, neutral): “Toical R eiew (2)2 [113] Hybrid ReLU (conv) C2I—Spect 4–30 Hz MR MW 91.”
Making Sense of Spatio-Temporal Preserving Representations for EEG-Based Human Intention Recognition (2020, methodology): “8) EEG-image + RCNN [17]: EEG-images are generated by complex preprocessing steps including FFT conversion, data compression, and interpolation before fed into an RCNN on the cognitive load classiﬁcation task.”
EEG-Based Brain-Computer Interfaces Using Motor-Imagery: Techniques and Challenges (2019, methodology): “This approach leads to a lower dimensionality input than the image-based approach in [56].”
Targeting EEG/LFP Synchrony with Neural Nets (2017, methodology): “This includes Restricted Boltzmann Machines and Deep Belief Networks [17, 36], CNNs [32, 29], and RNNs [2, 34].”
A novel deep learning approach for classification of EEG motor imagery signals (2017, methodology): “In a recent study [25], a new form of multi-dimensional features was proposed and employed in a recurrent-convolutional neural network architecture in order to model cognitive events from EEG data.”
Learning Robust Features using Deep Learning for Automatic Seizure Detection (2016, methodology): “To exploit the spatial locality existing in seizures we ﬁrst create an image-based representation of EEGs, integrating spatial domain knowledge (electrodes montage), using a method designed by Bashivan et al. (2016).”