You-Do, I-Learn: Discovering Task Relevant Objects and their Modes of Interaction from Multi-User Egocentric Video

By Dima Damen, Teesid Leelasawassuk, Osian Haines, Andrew D. Calway, Walterio W. Mayol-Cuevas. 2014

165 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Joint Discovery of Object States and Manipulation Actions (2017, neutral): “In [10], the authors use consistent manipulations to discover task relevant objects.”
Ego4D: Around the World in 3,000 Hours of Egocentric Video (2021, background): “[45] Dima Damen, Teesid Leelasawassuk, Osian Haines, Andrew Calway, and Walterio Mayol-Cuevas.”
Rescaling Egocentric Vision: Collection, Pipeline and Challenges for EPIC-KITCHENS-100 (2020, background): “Several leading egocentric datasets [35,36,37,38, 39] showcased the unique perspective and potential of first-person views for action recognition, particularly”
Scaling Egocentric Vision: The EPIC-KITCHENS Dataset (2018, methodology): “2 Related Datasets We compare EPIC-KITCHENS to four commonly-used [6, 13, 36, 8] and two recent [16, 41] egocentric datasets in Table 1, as well as six third-person activityrecognition datasets [14, 42, 56, 28, 44, 39] that focus on object-interaction activities.”
Cross-Task Weakly Supervised Learning From Instructional Videos (2019, background): “These have attracted a great deal of attention recently [1, 2, 19, 20, 24, 29, 35] since the co-occurrence of demonstrative visual actions and natural language enables many interesting tasks ranging from coreference resolution [19] to learning person-object interaction [2, 10].”
Trespassing the Boundaries: Labeling Temporal Bounds for Object Interactions in Egocentric Video (2017, methodology): “Previously, three works noted the challenge and difﬁ-culty in deﬁning temporal bounds for egocentric videos [29, 1, 37].”
You-Do, I-Learn: Egocentric unsupervised discovery of objects and their modes of interaction towards video-based guidance (2015, background): “The manuscript builds on our previous works towards offline [9] and online [8] discovery of task relevant objects, with additional novel contributions:”
Computer Vision and Image Understanding Computer Vision for Assistive Technologies (unknown): “One of the main application of First Person Vision systems is related memory augmentation ( Damen et al., 2014; Gurrin et al., 2014 ) (e.g., how to use an object?) and life logging ( Furnari et al., 2015; 2016; Gurrin et al., 2014; Ortis et al., 2016 ).”
Shaping embodied agent behavior with activity-context priors from egocentric video (2021, background): “More closely related to our work, visual affordance models derived from video can detect likely places for actions to occur [38, 18, 40], such as where to grasp a frying pan, and object saliency models can identify human-useable objects in egocentric video [14, 5, 20].”
Ego-Topo: Environment Affordances From Egocentric Video (2020, background): “This special viewpoint offers an array of interesting challenges, such as detecting gaze [40, 29], monitoring human-object interactions [4, 6, 51], creating daily life activity summaries [44, 39, 70, 43], or inferring the camera wearer’s identity or body pose [28, 33].”
MECCANO: A Multimodal Egocentric Dataset for Humans Behavior Understanding in the Industrial-like Domain (2022, background): “…from an egocentric perspective allows to build intelligent systems able to assist humans equipped with a camera (e.g., Microsoft Hololens 2 1 , Vuzix Blade 2 , Nreal Light 3 , etc.) in many contexts, including cultural sites [73, 11, 26], home scenarios [16] and industrial environments [10].”
Action Recognition From Single Timestamp Supervision in Untrimmed Videos (2019, methodology): “For BEOID and EPIC Kitchens, we notice a neat separation between the selected and discarded frames.”
The MECCANO Dataset: Understanding Human-Object Interactions from Egocentric Videos in an Industrial-like Domain (2020, background): “In the egocentric vision domain, in particular, previous investigations have considered the contexts of kitchens [13, 37, 17], as well as daily living activity at home and in offices [45, 57, 16, 44].”
Predicting the Future from First Person (Egocentric) Vision: A Survey (2021, background): “BEOID [92]: Dataset acquired by 5 subjects performing 34 activities.”
Seeing Invisible Poses: Estimating 3D Body Pose from Egocentric Video (2016, background): “Naturally, this makes them restricted to the arms and hands [4, 11, 12, 13, 20].”
Learning Affordance Landscapes for Interaction Exploration in 3D Environments (2020, background): “In computer vision, visual affordances are explored in various forms: predicting where to grasp an object from images and video [31, 32, 64, 38, 19, 62, 15, 5], inferring how people might use a space [48, 39] or tool [65], and priors for human body poses [26, 52, 58, 17].”