Benchmarking a Multimodal and Multiview and Interactive Dataset for Human Action Recognition

By Anan Liu, Ning Xu, Wei-zhi Nie, Yuting Su, Yongkang Wong, Mohan Kankanhalli. 2017

108 citations recorded by the source corpus.

Human action recognition is an active research area in both computer vision and machine learning communities. In the past decades, the machine learning problem has evolved from conventional single-view learning problem, to cross-view learning, cross-domain learning and multitask learning, where a large number of algorithms have been proposed in the literature. Despite having large number of action recognition datasets, most of them are designed for a subset of the four learning problems, where the comparisons between algorithms can further limited by variances within datasets, experimental configurations, and other factors. To the best of our knowledge, there exists no dataset that allows concurrent analysis on the four learning problems. In this paper, we introduce a novel multimodal and multiview and interactive (<inline-formula> <tex-math notation="LaTeX">$\text{M}^{{2}}\text{I}$ </tex-math></inline-formula>) dataset, which is designed for the evaluation of human action recognition methods under all four scenarios. This dataset consists of 1760 action samples from 22 action categories, including nine person-person interactive actions and 13 person-object interactive actions. We systematically benchmark state-of-the-art approaches on <inline-formula> <tex-math notation="LaTeX">$\text{M}^{{2}}\text{I}$ </tex-math></inline-formula> dataset on all four learning problems. Overall, we evaluated 13 approaches with nine popular feature and descriptor combinations. Our comprehensive analysis demonstrates that <inline-formula> <tex-math notation="LaTeX">$\text{M}^{{2}}\text{I}$ </tex-math></inline-formula> dataset is challenging due to significant intraclass and view variations, and multiple similar action categories, as well as provides solid foundation for the evaluation of existing state-of-the-art algorithms.

Subfields: Artificial Intelligence, Biomedical Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Biomedical Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Improving Interpretability of Deep Neural Networks with Semantic Information (2017, methodology): “[20, 21] proposed one original method for joint human action modeling and grouping, which can provide comprehensive information for video caption modeling and explicitly beneﬁt understanding what happens in the given video.”
Scene Flow to Action Map: A New Representation for RGB-D Based Action Recognition with Convolutional Neural Networks (2017, methodology): “For ChaLearn LAP IsoGD Dataset, in order to minimize the interference of the background, it is assumed that the background in the histogram of depth maps occupies the last peak representing far distances.”
Deep multi-view learning methods: A review (2021, background): “M2I[206] Video 2 22 1,760 2017 Typical challenging multi-view action dataset; Multi-modal and multi-view interactive human actions collected in dark, bright, and cluttered environments; Modalities, including RGB, depth, and skeleton, are simultaneously captured from front and side views by two stati”