Modeling Actions through State Changes

By Alireza Fathi, James M. Rehg. 2013

157 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Biomedical Engineering, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Biomedical Engineering

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Joint Discovery of Object States and Manipulation Actions (2017, methodology): “This observation has been used to design action models in [16, 17, 45].”
Ego4D: Around the World in 3,000 Hours of Egocentric Video (2021, background): “Relation to existing tasks Limited prior work considers object state change in photos [102,164] or video [8,68,242]; Ego4D is the first video benchmark dedicated to the task of understanding object state changes.”
Delving into Egocentric Actions (2015, neutral): “7% in GTEA Gaze+ over the state-of-the-art methods [9, 7, 37].”
Actions ~ Transformations (2015, background): “We differ from [5] in that we learn representations which enable explicit encoding of actions as transformations.”
Temporal Convolutional Networks for Action Segmentation and Detection (2016, methodology): “In the literature, our task goes by the name action segmentation [7, 8, 6, 15, 29, 16, 10] or action detection [28, 20, 21, 25].”
Visual Semantic Planning Using Deep Successor Representations (2017, background): “One possible way is to learn these from still images or videos [12, 51, 52].”
Asynchronous Temporal Fields for Action Recognition (2016, background): “In a similar vein, some works have also tried modeling activities as transformations [58] or state changes [5].”
A Survey on Activity Detection and Classification Using Wearable Sensors (2017, unknown): “[115] or chest-mounted cameras [111], [124].”
Analysis of Hand Segmentation in the Wild (2018, methodology): “3) GTEA dataset: Similar to [6] and [7], we also used images from subjects 1, 3 and 4 for training, and subject 2 for testing.”
Physical Causality of Action Verbs in Grounded Language Understanding (2016, background): “A recent trend in computer vision has started looking into intermediate representations beyond lower-level visual features for action recognition, for example, by incorporating object affordances (Koppula et al., 2013) and causality between actions and objects (Fathi and Rehg, 2013).”
Action Modifiers: Learning From Adverbs in Instructional Videos (2019, methodology): “Narrations are used to learn steps of complex tasks [1, 18, 30, 42, 45, 62], and more recently for video retrieval [33], visual grounding [17, 19], action segmentation [60] and learning actions through object state changes [2, 14].”
MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentation (2019, methodology): “While traditional approaches use a sliding window approach with non-maximum suppression [22, 11], Fathi and Rehg [7] model actions based on the change in the state of objects and materials.”
You-Do, I-Learn: Discovering Task Relevant Objects and their Modes of Interaction from Multi-User Egocentric Video (2014, background): “Common approaches in egocentric vision include i) segmenting the area surrounding the user’s hand [9, 10, 23], ii) extracting foreground regions through frame stabilisation or scene planarity assumptions [29, 36] or iii) detecting ‘object-like’ regions [25].”
Segmental Spatiotemporal CNNs for Fine-Grained Action Segmentation (2016, methodology): “Object-centric methods (e.g. [5,6,30]), ﬁrst detect the identity and location of objects in an image.”
Jointly Learning Energy Expenditures and Activities Using Egocentric Multimodal Signals (2017, background): “[23, 20] reason about state changes in household objects, and [17, 54] reason about human-object interactions.”
An end-to-end generative framework for video segmentation and recognition (2015, methodology): “In [7], activity models were based on the detection of changes in state-specific regions of interest (e.”
Egocentric Future Localization (2016, background): “This characteristics provides a powerful cue to understand human behaviors [6, 8, 15, 34, 36].”
VRKitchen: an Interactive 3D Virtual Environment for Task-oriented Learning (2019, background): “In the process of solving a task in a dynamic environment, the appearance of the same object may change dramatically as a result of actions [14, 19, 30].”
KrishnaCam: Using a longitudinal, single-person, egocentric dataset for scene understanding tasks (2016, background): “The two peaks of the graph (steep rises in the amount of novel visual data) correspond to days the student spent outside his home city.”
Going Deeper into First-Person Activity Recognition (2016, methodology): “We report comparative results on two subsets used in previous work [8, 20, 9]: 71 classes and 61 classes.”