First-Person Activity Forecasting with Online Inverse Reinforcement Learning

By Nicholas Rhinehart, Kris M. Kitani. 2016

149 citations recorded by the source corpus.

We address the problem of incrementally modeling and forecasting long-term goals of a first-person camera wearer: what the user will do, where they will go, and what goal they seek. In contrast to prior work in trajectory forecasting, our algorithm, DARKO, goes further to reason about semantic states (will I pick up an object?), and future goal states that are far in terms of both space and time. DARKO learns and forecasts from first-person visual observations of the user’s daily behaviors via an Online Inverse Reinforcement Learning (IRL) approach. Classical IRL discovers only the rewards in a batch setting, whereas DARKO discovers the states, transitions, rewards, and goals of a user from streaming data. Among other results, we show DARKO forecasts goals better than competing methods in both noisy and ideal settings, and our approach is theoretically and empirically no-regret.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Reinforcement Learning

Citations in the corpus

Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language (2022, background): “In the research community, activity forecasting has been often formulated as an extension of action classification, tracking, or feature generation: Given a sequence of image frames, they directly predict a few categorized actions (Ryoo 2011; Hoai and De la Torre 2014; Rhinehart and Kitani 2017), hu”
From Recognition to Cognition: Visual Commonsense Reasoning (2018, neutral): “Commonsense Reasoning Our task unifies work involving reasoning about commonsense phenomena, such as physics [54, 84], social interactions [2, 77, 12, 27], procedure understanding [91, 3] and predicting what might happen next in a video [74, 17, 92, 78, 18, 64, 85].”
Deep Imitative Models for Flexible Inference, Planning, and Control (2018, methodology): “Other methods include inverse reinforcement learning to ﬁt a probabilistic reward model to human demonstrations using the principle of maximum entropy (Ziebart et al., 2008; Sadigh et al., 2016; Rhinehart & Kitani, 2017).”
One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learning (2018, methodology): “For example, Rhinehart and Kitani [39] and Tow et al.”
Charades-Ego: A Large-Scale Dataset of Paired Third and First Person Videos (2018, background): “In recent years, the field of egocentric action understanding [4, 5, 6, 8, 3, 7] has exploded due to a variety of applications, including augmented and virtual reality.”
Rolling-Unrolling LSTMs for Action Anticipation from First-Person Video (2020, background): “Rhinehart and Kitani [63] used inverse reinforcement learning to anticipate future locations, objects and activities from egocentric video.”
What Would You Expect? Anticipating Egocentric Actions With Rolling-Unrolling LSTMs and Modality Attention (2019, background): “Anticipation in First Person Vision Past works on anticipation from egocentric visual data have investigated different problems and considered different evaluation frameworks [5, 10, 17, 42, 45, 47, 50, 53, 65, 68].”
Learning Real-World Robot Policies by Dreaming (2018, background): “[20] K. M. Kitani, B. D. Ziebart, J. A. Bagnell, and M. Hebert, “Activity forecasting,” in ECCV, 2012.”
The EPIC-KITCHENS Dataset: Collection, Challenges and Baselines (2020, background): “Previous works have investigated different anticipation tasks from an egocentric perspective, including predicting future localisation [62], next-active objects [63] or forecasting actions [64].”
When will you do what? - Anticipating Temporal Occurrences of Activities (2018, methodology): “[21] model the human behavior with a Markov Decision Process (MDP) and use inverse reinforcement learning to infer all elements of the MDP and the reward function from first person vision videos.”
AVID: Learning Multi-Stage Tasks via Pixel-Level Translation of Human Videos (2019, methodology): “To handle human videos, some prior meth-ods have used explicit pose and object detection [11]–[13], predictive modeling [14], [15], context translation [16], [17], and learning reward representations [18], [19].”
Ego-Exo: Transferring Visual Representations from Third-person to First-person Videos (2021, background): “Egocentric video understanding The unique viewpoint in egocentric video presents interesting research challenges including action recognition and anticipation [77, 21, 59, 1], daily life summary generation [36, 74], inferring body pose [32, 51], and estimating gaze [37, 31].”
Human Hands as Probes for Interactive Object Understanding (2021, background): “This can be in the form of learning a) where and how to grasp [9, 20, 25, 26, 28, 33, 34, 37, 41, 49], b) state classifiers [24], c) interaction hotspots [14, 40, 42, 59], d) spatial priors for action sites [44], e) object articulation modes [11,36], f) reward functions [27,29,48,50], g) functional ”
Few-Shot Goal Inference for Visuomotor Learning and Planning (2018, background): “Other works have sought to learn objectives by training a classiﬁer [12, 13, 14] or reward function [10, 24, 11], including in the framework of inverse RL [25, 7, 8, 26, 27, 9, 28, 29, 30].”
Multi-Stream Deep Neural Networks for RGB-D Egocentric Action Recognition (2019, unknown): “[78] explored the tasks of the first-person”
Future Person Localization in First-Person Videos (2017, background): “Recently, some work has focused on activity recognition [7, 22, 23, 28], activity forecasting [6, 9, 26, 31], person identification [11], gaze anticipation [45] and grasp recognition [3, 4, 21, 35].”
Leveraging Uncertainty to Rethink Loss Functions and Evaluation Measures for Egocentric Action Anticipation (2018, methodology): “[4] and Rhinehart and Kitani [25] proposed methods to predict future activities from first person video.”