Gaze360: Physically Unconstrained Gaze Estimation in the Wild

By Petr Kellnhofer, Adrià Recasens, Simon Stent, Wojciech Matusik, Antonio Torralba. 2019

494 citations recorded by the source corpus.

Understanding where people are looking is an informative social cue. In this work, we present Gaze360, a large-scale remote gaze-tracking dataset and method for robust 3D gaze estimation in unconstrained images. Our dataset consists of 238 subjects in indoor and outdoor environments with labelled 3D gaze across a wide range of head poses and distances. It is the largest publicly available dataset of its kind by both subject and variety, made possible by a simple and efficient collection method. Our proposed 3D gaze model extends existing models to include temporal information and to directly output an estimate of gaze uncertainty. We demonstrate the benefits of our model via an ablation study, and show its generalization performance via a cross-dataset evaluation against other recent gaze benchmark datasets. We furthermore propose a simple self-supervised approach to improve cross-dataset domain adaptation. Finally, we demonstrate an application of our model for estimating customer attention in a supermarket setting. Our dataset and models will be made available at http://gaze360.csail.mit.edu.

Subfields: Computer Vision and Pattern Recognition, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Human-Computer Interaction

Citations in the corpus

Ego4D: Around the World in 3,000 Hours of Egocentric Video (2021, background): “Mutual gaze estimation [54,150–152,172,176] and gaze following [37,65,111,186] are also relevant.”
Oops! Predicting Unintentional Action in Video (2019, background): “Moreover, performance is comparatively low when the person in the video has limited visibility, such as due to occlusions, which motivates further work in gaze estimation [47, 29], especially in video.”
Dual Attention Guided Gaze Target Detection in the Wild (2021, methodology): “Besides, we evaluate our 3D gaze estimator on the Gaze360 dataset [22].”
ETH-XGaze: A Large Scale Dataset for Gaze Estimation under Extreme Head Pose and Gaze Variation (2020, methodology): “In [16], it is stated that the effective head pose range of Gaze360 is ±90◦ in horizontal direction and limited head poses in vertical direction.”