Discovering important people and objects for egocentric video summarization

By Yong Jae Lee, Joydeep Ghosh, Kristen Grauman. 2012

778 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language (2022, background): “, optical flows) [98, 99] as well as features from human gaze, hands, and objects [100, 101, 102, 103, 104, 105].”
Ego4D: Around the World in 3,000 Hours of Egocentric Video (2021, methodology): “Our dataset draws inspiration from prior ego-centric video data efforts [43,44,129,138,179,201,205,210], but makes significant advances in terms of scale, diversity, and realism.”
Delving into Egocentric Actions (2015, neutral): “By taking advantage of a point-of-view camera, there have been several recent advances in First-Person Vision (FPV) [14], also known as egocentric vision, such as video summarization [20], video stabilization [16], object recognition [29] and action and activity recognition [33, 6, 27].”
Scaling Egocentric Vision: The EPIC-KITCHENS Dataset (2018, background): “3 Active Object Bounding Box Annotations The narrated nouns correspond to objects relevant to the action [21], [36].”
Unsupervised Semantic Parsing of Video Collections (2015, methodology): “Summarizing videos is particularly important for long sequences like ego-centric videos and news reports [35, 38, 50]; however, these methods mostly rely on characteristics of the application and do not generalize.”
Watch and learn: Semi-supervised learning of object detectors from videos (2015, background): “We determine good detections (removing conﬁ-dent false positives) which serve as starting points for short-term tracking.”
Charades-Ego: A Large-Scale Dataset of Paired Third and First Person Videos (2018, background): “In fact, our dataset is one of the largest first-person datasets available [2, 4, 6, 1], has significantly more diversity (112 actors in many rooms), and most importantly, is the only large-scale dataset to offer pairs of first and third-person views that we can learn from.”
Egocentric Video-Language Pretraining (2022, background): “5 M HowTo100M [10] instructional 134 K 136 M 136 M 3rd-person view Charades-Ego [17] home 34 30 K 30 K UT-Ego [18] diverse 37 11 K 11 K Disneyworld [19] disneyland 42 15 K 15 K EPIC-KITCHENS-100 [20] kitchen 100 90 K 90 K EgoClip diverse 2 .”
A Survey on Activity Detection and Classification Using Wearable Sensors (2017, background): “including predicting important objects and people [191], [192], extracting frames that might be a snapshot from ego-centric”
TVSum: Summarizing web videos using titles (2015, background): “, salient objects in sports [17], and salient regions in egocentric videos [27, 32].”
Semantic Pooling for Complex Event Analysis in Untrimmed Videos (2016, background): “Prioritizing objects according to saliency [9] is ubiquitous in visual tasks such as segmentation [10] and video summarization [11].”
Creating Summaries from User Videos (2014, methodology): “Probably the most related work to ours are the recent works done at UT Austin [18] [22].”
Listen to Look: Action Recognition by Previewing Audio (2019, background): “Video summarization work also aims to select keyframes or clips [36, 25, 38, 75], but with the purpose of conveying the gist of the video to a human viewer.”
VIP: Finding important people in images (2015, background): “This reasoning is across a corpus of photos (each containing a person of interest), and the objective is to assign an importance score to each image.”
Video Segmentation by Tracking Many Figure-Ground Segments (2013, background): “These advances have made unsupervised segmentation applicable to difficult high-level tasks, such as semantic segmentation [23, 2, 10] and video saliency reasoning [19].”
A Weighted Sparse Sampling and Smoothing Frame Transition Approach for Semantic Fast-Forward First-Person Videos (2018, background): “[11] exploit interaction level, gaze, and object detection frequency as egocentric properties to create a storyboard of keyframes with important people and objects.”
Making 360° Video Watchable in 2D: Learning Videography for Click Free Viewing (2017, background): “Video summarization Video summarization aims to generate a concise representation for a video by removing temporal redundancy while preserving the important events [8, 14–16, 24, 27, 32, 33, 38, 42].”
Unsupervised Video Summarization with Adversarial LSTM Networks (2017, background): “Depending on the application, there are various distinct definitions of video summarization [30, 27, 28, 1, 40, 37, 6, 20, 18, 14, 25, 12].”
The EPIC-KITCHENS Dataset: Collection, Challenges and Baselines (2020, background): “The narrated nouns correspond to objects relevant to the action [22], [43].”
Summarization of Egocentric Videos: A Comprehensive Survey (2017, methodology): “Finally, importance may be estimated with supervised learning [23], [25], [26]; inferred from impersonal cues such as saliency [17], people and object interaction [10], [12], [16], [17], [19], and location [13]; or predicted from the wearer attention patterns, using sensors [14], [15], [22] or motio”