FineGym: A Hierarchical Video Dataset for Fine-Grained Action Understanding

By Dian Shao, Yue Zhao, Bo Dai, Dahua Lin. Computer Vision and Pattern Recognition, 2020

441 citations recorded by the source corpus.

On public benchmarks, current action recognition techniques have achieved great success. However, when used in real-world applications, e.g. sport analysis, which requires the capability of parsing an activity into phases and differentiating between subtly different actions, their performances remain far from being satisfactory. To take action recognition to a new level, we develop FineGym, a new dataset built on top of gymnasium videos. Compared to existing action recognition datasets, FineGym is distinguished in richness, quality, and diversity. In particular, it provides temporal annotations at both action and sub-action levels with a three-level semantic hierarchy. For example, a “balance beam” activity will be annotated as a sequence of elementary sub-actions derived from five sets: “leap-jump-hop”, “beam-turns”, “flight-salto”, “flight-handspring”, and “dismount”, where the sub-action in each set will be further annotated with finely defined class labels. This new level of granularity presents significant challenges for action recognition, e.g. how to parse the temporal structures from a coherent action, and how to distinguish between subtly different action classes. We systematically investigates different methods on this dataset and obtains a number of interesting findings. We hope this dataset could advance research towards action understanding.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/f5c35edc9cf622c7dd7e19ce6fbd5d563557de5b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Libin Liu
2. Wenjie Zhu
3. Wayne Wu
4. Ziwei Liu
5. Mingyuan Zhang
6. Xiaoxuan Ma
7. Zhaoyang Liu
8. Liang Pan

In Computer Vision and Pattern Recognition

1. Wayne Wu
2. Zhaoyang Liu
3. Liang Pan
4. Mingyuan Zhang
5. Libin Liu
6. Wenjie Zhu
7. Fangzhou Hong
8. Ziwei Liu

Citations in the corpus

• VideoMamba: State Space Model for Efficient Video Understanding (2024, neutral): “Other datasets [22, 27, 31, 35, 49, 63] provide annotated activity videos for action localization, fostering deeper research into human activities.”
• MotionDiffuse: Text-Driven Human Motion Generation With Diffusion Model (2022, background): “, 2019), finegrained action understanding (Gu et al., 2018; Shao et al., 2020; Chung et al., 2021) and 3D data (Liu et al.”
• MotionBERT: A Unified Perspective on Learning Human Motion Representations (2022, background): “Action recognition datasets provide annotations of the action semantics, but they either contain no human pose labels [16, 95] or feature limited motion of daily activities [63, 64, 93].”