Learning to Predict Gaze in Egocentric Video

By Yin Li, Alireza Fathi, James M. Rehg. 2013

281 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition, Urban Studies, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Urban Studies

In Human-Computer Interaction

Citations in the corpus

Ego4D: Around the World in 3,000 Hours of Egocentric Video (2021, neutral): “[135] Yin Li, Alireza Fathi, and James M.”
Delving into Egocentric Actions (2015, positive): “Our previous work [22] models the coordination of hand, head and eye, which is then used to predict egocentric gaze from hand pose and head movement.”
A Survey on Activity Detection and Classification Using Wearable Sensors (2017, background): “The groundwork for using visual attention and saliency can be linked to other works, including [196], [117], and [197].”
Gaze Prediction in Dynamic 360° Immersive Videos (2018, unknown): “of movement [39] and hand location [26].”
Learning to Look Around: Intelligently Exploring Unseen Environments for Unknown Tasks (2017, background): “Saliency and attention: Previous work studies the question of “where to look” to prioritize portions of already captured image/video data, so as to reserve computation for the most salient regions or block out distractors [1, 5, 8, 23, 41, 47, 52, 62, 71], or to predict the gaze or preference of a h”
Ego-Topo: Environment Affordances From Egocentric Video (2020, background): “This special viewpoint offers an array of interesting challenges, such as detecting gaze [40, 29], monitoring human-object interactions [4, 6, 51], creating daily life activity summaries [44, 39, 70, 43], or inferring the camera wearer’s identity or body pose [28, 33].”
Multi-Stream Deep Neural Networks for RGB-D Egocentric Action Recognition (2019, background): “tion [4]–[6], person re-identification [7] and gaze analysis [8].”
H2O: Two Hands Manipulating Objects for First Person Interaction Recognition (2021, background): “It is also common to extract features with CNNs and leverage additional 2D cues related to motion, hand location, object location or object class, in isolation [3, 46, 53] or jointly [19, 21, 25, 67, 74].”
Learning image representations equivariant to ego-motion (2015, methodology): “Recent methods use ego-motion cues to separate foreground and background [23] [33] or infer the firstperson gaze [34] [16].”
Ego-Exo: Transferring Visual Representations from Third-person to First-person Videos (2021, background): “Egocentric video understanding The unique viewpoint in egocentric video presents interesting research challenges including action recognition and anticipation [77, 21, 59, 1], daily life summary generation [36, 74], inferring body pose [32, 51], and estimating gaze [37, 31].”
Going Deeper into First-Person Activity Recognition (2016, background): “Besides hand location, hand pose and shape is also important to estimate the manipulation points as shown in [19].”
You-Do, I-Learn: Discovering Task Relevant Objects and their Modes of Interaction from Multi-User Egocentric Video (2014, methodology): “Gaze has been successfully used in to assist action recognition [11, 24] or supervised object recognition [8, 34].”
You-Do, I-Learn: Egocentric unsupervised discovery of objects and their modes of interaction towards video-based guidance (2015, methodology): “Gaze has been successfully used to assist action and activity recognition [12, 35, 44, 38] and supervised object recognition [53, 11].”
Temporal Perception and Prediction in Ego-Centric Video (2015, background): “There has been recent work on object recognition [12, 31], activity recognition [30, 9, 33], gaze prediction [25], and interaction detection [10] in first person video.”
See the Forest for the Trees: Joint Spatial and Temporal Recurrent Neural Networks for Video-Based Person Re-identification (2017, unknown): “Furthermore, videos contain richer information than a single image [19, 20], which is beneficial for identifying a person under complex conditions, including occlusions, and the changes of illumination and viewpoint.”
FaceShifter: Towards High Fidelity And Occlusion Aware Face Swapping (2019, methodology): “Occlusion images are randomly sampled from the EgoHands [30], GTEA Hand2K [31, 32, 33] and object renderings from ShapeNet [34].”
In the Eye of Beholder: Joint Learning of Gaze and Actions in First Person Video (2018, methodology): “Our previous work [18] estimated egocentric gaze using hand and head cues.”
Jointly Learning Energy Expenditures and Activities Using Egocentric Multimodal Signals (2017, background): “Adding mid-level cues such as face, gaze, and hands has also been investigated by [37, 19, 18, 49, 6].”