Detecting Engagement in Egocentric Video

By Yu-Chuan Su, Kristen Grauman. 2016

57 citations recorded by the source corpus.

In a wearable camera video, we see what the camera wearer sees. While this makes it easy to know roughly Open image in new window , it does not immediately reveal Open image in new window . Specifically, at what moments did his focus linger, as he paused to gather more information about something he saw? Knowing this answer would benefit various applications in video summarization and augmented reality, yet prior work focuses solely on the “what” question (estimating saliency, gaze) without considering the “when” (engagement). We propose a learning-based approach that uses long-term egomotion cues to detect engagement, specifically in browsing scenarios where one frequently takes in new visual information (e.g., shopping, touring). We introduce a large, richly annotated dataset for ego-engagement that is the first of its kind. Our approach outperforms a wide array of existing methods. We show engagement can be detected well independent of both scene appearance and the camera wearer’s identity.

Subfields: Computer Vision and Pattern Recognition, Sociology and Political Science

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Sociology and Political Science

Citations in the corpus

Ego4D: Around the World in 3,000 Hours of Egocentric Video (2021, methodology): “Most relevant to our work are those containing unscripted daily life activity, which includes EPIC-Kitchens [43,44], UT Ego [129,210], Activities of Daily Living (ADL) [179], and the Disney dataset [66].”
Next-active-object prediction from egocentric videos (2017, background): “Towards this direction, researchers have investigated methods to understand the user’s environment [3–8], model his attention [9,10], categorize his motion [11,12], summarize the acquired video [13,14], recognize performed activities [15–18], and provide assistance [1,19].”
EgoVQA - An Egocentric Video Question Answering Benchmark Dataset (2019, background): “More and more people are using wearable cameras for various applications such as lifelogging [13,22,30], children behavior study [8,9], sports video analysis [11,12], event detection [50], etc.”