LAEO-Net: Revisiting People Looking at Each Other in Videos

By Manuel Jesús Marín-Jiménez, Vicky Kalogeiton, Pablo Medina-Suarez, Andrew Zisserman. 2019

76 citations recorded by the source corpus.

Capturing the ‘mutual gaze’ of people is essential for understanding and interpreting the social interactions between them. To this end, this paper addresses the problem of detecting people Looking At Each Other (LAEO) in video sequences. For this purpose, we propose LAEO-Net, a new deep CNN for determining LAEO in videos. In contrast to previous works, LAEO-Net takes spatio-temporal tracks as input and reasons about the whole track. It consists of three branches, one for each character’s tracked head and one for their relative position. Moreover, we introduce two new LAEO datasets: UCO-LAEO and AVA-LAEO. A thorough experimental evaluation demonstrates the ability of LAEO-Net to successfully determine if two people are LAEO and the temporal window where it happens. Our model achieves state-of-the-art results on the existing TVHID-LAEO video dataset, significantly outperforming previous approaches.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Ego4D: Around the World in 3,000 Hours of Egocentric Video (2021, background): “[150] Manuel J Marin-Jimenez, Vicky Kalogeiton, Pablo MedinaSuarez, and Andrew Zisserman.”
Detecting Attended Visual Targets in Video (2020, methodology): “Given that our model does not have a head detection module as in [13], we trained a SSD-based [27] head detector in the same way as [29] to automatically generate the input head positions.”
Dual Attention Guided Gaze Target Detection in the Wild (2021, methodology): “For these reasons, gaze analysis has widely been used in neurophysiology studies [36, 10], relevant saliency prediction [8, 34], and social awareness tracking [7, 29, 30].”
Boosting Image-based Mutual Gaze Detection using Pseudo 3D Gaze (2020, methodology): “• We show that the proposed approach outperforms the state-of-the-art mutual gaze detection approach of (Marin-Jimenez et al. 2019) 1 on three image datasets.”