Learning to Recognize Daily Actions Using Gaze

By Alireza Fathi, Yin Li, James M. Rehg. 2012

444 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Human-Computer Interaction

Citations in the corpus

Rescaling Egocentric Vision: Collection, Pipeline and Challenges for EPIC-KITCHENS-100 (2020, background): “Several leading egocentric datasets [35,36,37,38, 39] showcased the unique perspective and potential of first-person views for action recognition, particularly”
Delving into Egocentric Actions (2015, neutral): “In contrast, state-of-the-art egocentric action recognition methods [6] [27] [7] rely mainly on an object-centric representation for discriminating action categories.”
Modeling Temporal Dynamics and Spatial Configurations of Actions Using Two-Stream Recurrent Neural Networks (2017, background): “Human action recognition [2] has become an active area in computer vision and there are many important research problems, such as event recognition [23], group based activities recognition [27], human object interactions [15] and activities in egocentric videos [29, 11].”
A Comprehensive Survey of Deep Learning for Image Captioning (2018, background): “Thus, gaze mechanisms have already shown their potential performances in eye-based user modeling [18, 35, 109, 122, 124], object localization [100] or recognition [67] and holistic scene understanding [158, 160].”
Rolling-Unrolling LSTMs for Action Anticipation from First-Person Video (2020, background): “Fathi et al. [37] employed eye gaze measurements to recognize egocentric actions.”
Summarization of Egocentric Videos: A Comprehensive Survey (2017, methodology): “of datasets such as the following: CMU-MMAC [51], recorded in a staged kitchen; GTEA [52] and GTEA-gaze [50], which contain very specific videos, sometimes staged and short; UEC”
Recognizing Micro-Actions and Reactions from Paired Egocentric Videos (2016, background): “In this way, egocentric videos are advantageous from a sensing perspective since the head motion and hand motion of camera wearers are often observed clearly in such videos, making it possible to perform various forms of first-person action recognition [7, 9, 13, 18, 25, 27, 30].”
What Would You Expect? Anticipating Egocentric Actions With Rolling-Unrolling LSTMs and Modality Attention (2019, background): “Action Recognition Our work is related to previous research on action recognition from third person vision [7, 13, 14, 30, 34, 35, 49, 57, 59, 60, 61, 67] and first person vision [11, 12, 36, 37, 38, 44, 48, 52, 54, 55, 56].”
Temporal Perception and Prediction in Ego-Centric Video (2015, background): “There are several existing egocentric datasets [12, 30, 11, 10, 24] designed for a variety of purposes, such as object recognition, activity recognition, social interaction detection and video summarization.”
You-Do, I-Learn: Egocentric unsupervised discovery of objects and their modes of interaction towards video-based guidance (2015, methodology): “Gaze has been successfully used to assist action and activity recognition [12, 35, 44, 38] and supervised object recognition [53, 11].”
Who is Mistaken? (2016, unknown): “[5, 28, 7, 20, 10].”
Going Deeper into First-Person Activity Recognition (2016, methodology): “We report results on two subsets used in previous works [20, 6]: 40 classes and 25 classes.”
H+O: Unified Egocentric Recognition of 3D Hand-Object Poses and Interactions (2019, background): “In addition to motion, features based on gaze information [13], head motion [33], and, recently, CNN-learned features [35] have been proposed by a number of studies.”
The EPIC-KITCHENS Dataset: Collection, Challenges and Baselines (2020, background): “[21], [22], [23], [24], [25] have been significantly smaller in size”
You-Do, I-Learn: Discovering Task Relevant Objects and their Modes of Interaction from Multi-User Egocentric Video (2014, methodology): “Gaze has been successfully used in to assist action recognition [11, 24] or supervised object recognition [8, 34].”
Jointly Learning Energy Expenditures and Activities Using Egocentric Multimodal Signals (2017, background): “5 - 640×480 Head X - - Indoor Object GTEA Gaze [19] 14 17 1.”
In the Eye of the Beholder: Gaze and Actions in First Person Video (2020, methodology): “dataset—a major expansion of our previous GTEA Gaze+ dataset [2], [19].”
Trespassing the Boundaries: Labeling Temporal Bounds for Object Interactions in Egocentric Video (2017, methodology): “…Bounds in Egocentric Datasets Compared to third person action recognition (e.g. 101 action classes in [28] and 157 action classes in [26]), egocentric datasets have a smaller number of classes (5-44 classes [2, 7, 6, 1, 15, 21, 37]), with considerable ambiguities (e.g. ‘turn on’ vs ‘turn off’ tap).”