DEEP eye contact detector: Robust eye contact bid detection using convolutional neural network
By Yu Mitsuzumi, Atsushi Nakazawa, Toyoaki Nishida. 2017
20 citations recorded by the source corpus.
Eye contact (mutual gaze) is a foundation of human communication and social interactions; therefore, it is studied in many fields such as psychology, social science, and medicine. To support these application of eye-contact detection, much effort has been made for automated eye-contact detection by using image recognition techniques; however, they are difficult to use when facial-landmark tracking is not possible due to facial occlusions. To solve this issue, this paper proposes robust algorithms of detecting eye contacts leveraged using deep neural net to find the eye-contact skills needed for care-givers in dementia care. Since our algorithms do not depend on facial-landmark tracking and only use the images around the eyes, they are robust against facial occlusions and/or image noise. We prepared two eye contact facial image datasets and confirmed the performance of the proposed algorithms. The results show that our method are robust against facial occlusion in which a person is wearing a facemask or the person’s face is partially outside the camera viewing area. This study shows the potential to be able to obtain the eye-contact status only from the images around eyes.
Subfields: Computer Vision and Pattern Recognition, Human-Computer Interaction
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Kristen Grauman
- Luc Van Gool
- Jitendra Malik
- Trevor Darrell
- Andrew Zisserman
- Cordelia Schmid
- Dacheng Tao
- Ming-Hsuan Yang
In Human-Computer Interaction
- Kristen Grauman
- Antonino Furnari
- Giovanni Maria Farinella
- Miao Liu
- Tushar Nagarajan
- Kris M. Kitani
- James M. Rehg
- Antonio Torralba
Citations in the corpus
- Ego4D: Around the World in 3,000 Hours of Egocentric Video (2021, background): “The LAM task is most closely related to prior work on eye contact detection in ego-video [36,159], but addresses more diverse and challenging scenarios.”
- Detecting Attended Visual Targets in Video (2020, background): “Prior works have also addressed the detection of eye contact and mutual gaze in the context of dementia care [33, 37] and autism [50, 9].”