Action recognition in RGB-D egocentric videos

By Yansong Tang, Yi Tian, Jiwen Lu, Jianjiang Feng, Jie Zhou. 2017

51 citations recorded by the source corpus.

In this paper, we investigate the problem of action recognition in RGB-D egocentric videos. These self-generated and embodied videos provide richer semantic cues than the conventional videos captured from the third-person view for action recognition. Moreover, they contain both appearance information and 3D structure of the scenes from the RGB modality and depth modality respectively. Motivated by these advantages, we first collect a video-based RGB-D egocentric dataset (THU-READ) with diverse types of daily-life actions. Then we evaluate several approaches including hand-crafted features and deep learning methods on THU-READ. To improve the performance, we further develop a tri-stream convolutional network (TCNet) method, which learns to exploit the fuse with both the RGB and depth modalities for action recognition. Experimental results show that our model achieves competitive performance with state-of-the-art methods.

Subfields: Biomedical Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Fine-Grained Egocentric Hand-Object Segmentation: Dataset, Model, and Applications (2022, methodology): “A major characteristic is that our dataset contains very diverse hand-object interaction activities and scenarios, where frames are sparsely sampled from nearly 1,000 videos in Ego4D [16], EPIC-KITCHEN [8], THU-READ [68], and from our own collected GoPro videos.”
Egocentric Hand Track and Object-Based Human Action Recognition (2019, methodology): “IEORD [29] EDSH [22] EgoHands [25] EGTEA+ [27] THU-READ [28] Combined Test EDSH 71 100 (0) 17 (49) 72.”
MECCANO: A Multimodal Egocentric Dataset for Humans Behavior Understanding in the Industrial-like Domain (2022, background): “…ﬁelds have been obtained thanks to the availability of public datasets [51, 60, 43, 7] considering different contexts such as kitchens [13, 15, 57, 17], home and ofﬁces [69, 89, 16, 67], different daily-living scenarios [41] and relying on different modalities such as depth [89] and gaze [57].”
The MECCANO Dataset: Understanding Human-Object Interactions from Egocentric Videos in an Industrial-like Domain (2020, methodology): “In the egocentric vision domain, in particular, previous investigations have considered the contexts of kitchens [13, 37, 17], as well as daily living activity at home and in offices [45, 57, 16, 44].”
H+O: Unified Egocentric Recognition of 3D Hand-Object Poses and Interactions (2019, unknown): “Motion and appearance cues are used in conjunction with depth in [63].”