It’s Written All Over Your Face: Full-Face Appearance-Based Gaze Estimation

By Xucong Zhang, Yusuke Sugano, Mario Fritz, Andreas Bulling. 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2016

513 citations recorded by the source corpus.

Eye gaze is an important non-verbal cue for human affect analysis. Recent gaze estimation work indicated that information from the full face region can benefit performance. Pushing this idea further, we propose an appearance-based method that, in contrast to a long-standing line of work in computer vision, only takes the full face image as input. Our method encodes the face image using a convolutional neural network with spatial weights applied on the feature maps to flexibly suppress or enhance information in different facial regions. Through extensive evaluation, we show that our full-face method significantly outperforms the state of the art for both 2D and 3D gaze estimation, achieving improvements of up to 14.3% on MPIIGaze and 27.7% on EYEDIAP for person-independent 3D gaze estimation. We further show that this improvement is consistent across different illumination conditions and gaze directions and particularly pronounced for the most challenging extreme head poses.

Subfields: Computer Vision and Pattern Recognition, Human-Computer Interaction

View Paper: https://www.semanticscholar.org/paper/7eb3ce75e6b26c664c23265ac8efdfacdaac1a3e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Human-Computer Interaction

Citations in the corpus

Controllable Continuous Gaze Redirection (2020, unknown): “l landmarks 755,370 ✓ Small Gaze Capture [17] ✓ ✗ ✗ 2D position on screen >2.5M ✗ Small SynthesEyes [37] ✗ ✓ ✗ Gaze vector 11,382 ✓ Full UnityEyes [34] ✗ ✓ ✗ Gaze vector 1,000,000 ✓ Full MPII Gaze [43, 44] ✓ ✗ ✗ Gaze vector 213,659 ✓ Small UT Multi-View [30] ✓ ✗ ✓ Gaze vector 1,152,000 ✓ Large Co”
Dual Attention Guided Gaze Target Detection in the Wild (2021, background): “Some practices [45, 38, 22, 33] approximate gaze by head pose, allowing for coverage over a wide range of head poses.”
MPIIGaze: Real-World Dataset and Deep Appearance-Based Gaze Estimation (2017, unknown): “the full face image as input, resulting again in significant performance improvements for both 2D and 3D gaze estimation [55].”
RT-GENE: Real-Time Eye Gaze Estimation in Natural Environments (2018, methodology): “face image Head pose vector 18,117 Varying X - All Eyediap [15] RGB-D Face + eye patches Gaze vector ≈62,500 80-120cm X X Frontal MPII Gaze [60,61] RGB Face + eye patches Gaze vector 213,659 40-60cm X X Frontal Columbia [52] RGB High res.”
Gaze360: Physically Unconstrained Gaze Estimation in the Wild (2019, methodology): “Support vector regression [28], random forests [11] and most recently deep learning [4, 12, 30, 31, 34] have been applied in this way.”
Boosting Image-based Mutual Gaze Detection using Pseudo 3D Gaze (2020, background): “(Zhang et al. 2017b) collected a 3D gaze estimation dataset from 15 users during their laptop usage over several months.”
DEEP eye contact detector: Robust eye contact bid detection using convolutional neural network (2017, methodology): “They also developed a DNN-based algorithm using a full-facial image without occlusions [26].”