A Semisupervised Recurrent Convolutional Attention Model for Human Activity Recognition

By Lina Yao, Dalin Zhang, Xianzhi Wang, Xiaojun Chang, Feiping Nie. IEEE Transactions on Neural Networks and Learning Systems, 2020

300 citations recorded by the source corpus.

Recent years have witnessed the success of deep learning methods in human activity recognition (HAR). The longstanding shortage of labeled activity data inherently calls for a plethora of semisupervised learning methods, and one of the most challenging and common issues with semisupervised learning is the imbalanced distribution of labeled data over classes. Although the problem has long existed in broad real-world HAR applications, it is rarely explored in the literature. In this paper, we propose a semisupervised deep model for imbalanced activity recognition from multimodal wearable sensory data. We aim to address not only the challenges of multimodal sensor data (e.g., interperson variability and interclass similarity) but also the limited labeled data and class-imbalance issues simultaneously. In particular, we propose a pattern-balanced semisupervised framework to extract and preserve diverse latent patterns of activities. Furthermore, we exploit the independence of multi-modalities of sensory data and attentively identify salient regions that are indicative of human activities from inputs by our recurrent convolutional attention networks. Our experimental results demonstrate that the proposed model achieves a competitive performance compared to a multitude of state-of-the-art methods, both semisupervised and supervised ones, with 10% labeled training data. The results also show the robustness of our method over imbalanced, small training data sets.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/81bf935b1049be72fb3b5140d0d3647b7c5bbd3f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Zhiwen Yu
2. Bin Guo
3. Lina Yao
4. Yunhao Liu
5. Dalin Zhang

In Computer Vision and Pattern Recognition

1. Zhiwen Yu
2. Bin Guo
3. Lina Yao
4. Yunhao Liu
5. Dalin Zhang

Citations in the corpus

• Deep Learning for Sensor-based Human Activity Recognition (2020, background): “[28] applied co-training with multiple classifiers on different modalities of the data.”