A multimodal approach to estimating vigilance using EEG and forehead EOG

By Wei-Long Zheng, Bao-Liang Lu. Journal of Neural Engineering, 2016

385 citations recorded by the source corpus.

Objective. Covert aspects of ongoing user mental states provide key context information for user-aware human computer interactions. In this paper, we focus on the problem of estimating the vigilance of users using EEG and EOG signals. Approach. The PERCLOS index as vigilance annotation is obtained from eye tracking glasses. To improve the feasibility and wearability of vigilance estimation devices for real-world applications, we adopt a novel electrode placement for forehead EOG and extract various eye movement features, which contain the principal information of traditional EOG. We explore the effects of EEG from different brain areas and combine EEG and forehead EOG to leverage their complementary characteristics for vigilance estimation. Considering that the vigilance of users is a dynamic changing process because the intrinsic mental states of users involve temporal evolution, we introduce continuous conditional neural field and continuous conditional random field models to capture dynamic temporal dependency. Main results. We propose a multimodal approach to estimating vigilance by combining EEG and forehead EOG and incorporating the temporal dependency of vigilance into model training. The experimental results demonstrate that modality fusion can improve the performance compared with a single modality, EOG and EEG contain complementary information for vigilance estimation, and the temporal dependency-based models can enhance the performance of vigilance estimation. From the experimental results, we observe that theta and alpha frequency activities are increased, while gamma frequency activities are decreased in drowsy states in contrast to awake states. Significance. The forehead setup allows for the simultaneous collection of EEG and EOG and achieves comparative performance using only four shared electrodes in comparison with the temporal and posterior sites.

Subfields: Cognitive Neuroscience, Human-Computer Interaction

View Paper: https://www.semanticscholar.org/paper/babe42b42bffd9029d7fef3d6c97ec0985fa6528

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

1. Lin Shu
2. Jason Teo
3. Dan Liao
4. Zhenqi Li
5. Ziyi Li
6. James Mountstephens
7. Mingyue Yang
8. Jinyan Xie

In Human-Computer Interaction

1. Xiangmin Xu

Citations in the corpus

• EEG-Based Spatio–Temporal Convolutional Neural Network for Driver Fatigue Evaluation (2019, neutral): “In [17], a multimodal approach was proposed to estimate driver fatigue by combin-”
• EEG-Based Emotion Recognition: A State-of-the-Art Review of Current Trends and Opportunities (2020, background): “As for video-related contents, SEED [124–126], DREAMER [127], and ASCERTAIN [107] do provide their video database either openly or upon request.”
• Transfer Learning for EEG-Based Brain–Computer Interfaces: A Review of Progress Made Since 2016 (2020, neutral): “There are many important BCI regression problems, e.g., driver drowsiness estimation [38]–[40], vigilance estimation [11], [12], [113], and user reaction time estimation [41], which were not adequately addressed in previous reviews.”
• A Review of Emotion Recognition Using Physiological Signals (2018, background): “The SEED database [135] contains EEG and eye movement of three different emotions (positive, neutral and negative).”