Ensembles of Deep LSTM Learners for Activity Recognition using Wearables

By Yu Guan, Thomas Plötz. Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies, 2017

510 citations recorded by the source corpus.

Recently, deep learning (DL) methods have been introduced very successfully into human activity recognition (HAR) scenarios in ubiquitous and wearable computing. Especially the prospect of overcoming the need for manual feature design combined with superior classification capabilities render deep neural networks very attractive for real-life HAR applications. Even though DL-based approaches now outperform the state-of-the-art in a number of recognition tasks, still substantial challenges remain. Most prominently, issues with real-life datasets, typically including imbalanced datasets and problematic data quality, still limit the effectiveness of activity recognition using wearables. In this paper we tackle such challenges through Ensembles of deep Long Short Term Memory (LSTM) networks. LSTM networks currently represent the state-of-the-art with superior classification performance on relevant HAR benchmark datasets. We have developed modified training procedures for LSTM networks and combine sets of diverse LSTM learners into classifier collectives. We demonstrate that Ensembles of deep LSTM learners outperform individual LSTM networks and thus push the state-of-the-art in human activity recognition using wearables. Through an extensive experimental evaluation on three standard benchmarks (Opportunity, PAMAP2, Skoda) we demonstrate the excellent recognition capabilities of our approach and its potential for real-life applications of human activity recognition.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/4b8446ea306d7d3f8218a8850158b0ae084a988e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Deep Learning for Sensor-based Human Activity Recognition (2020, methodology): “st a certain percentage of data in the training set. Another direction of solving the imbalance concern is to modify the model-building strategy instead of directly balancing the training dataset. In [49], Guan and Plötz utilized the F1-score rather than the conventional cross-entropy as the loss fu”
Deep Learning for Sensor-based Activity Recognition: A Survey (2017, methodology): “(Guan and Ploetz, 2017) Body-worn, object, ambient RNN ADL, smart home D01, D02, D04”
A Semisupervised Recurrent Convolutional Attention Model for Human Activity Recognition (2020, methodology): “Although some approaches [10], [16] have been proposed to solve the imbalance issue, their performance deteriorates due to the small data size in semisupervised learning.”
Deep PPG: Large-Scale Heart Rate Estimation with Convolutional Neural Networks (2019, background): “Therefore, since capturing more data variability during training, combining these models will lead to increased modelling capabilities [49].”
Zero-Effort Cross-Domain Gesture Recognition with Wi-Fi (2019, methodology): “Traditional approaches use cameras [16, 24, 42], wearable devices and phones [8, 17, 36] or sonar [22, 29, 48] as the sensing module.”
Widar3.0: Zero-Effort Cross-Domain Gesture Recognition With Wi-Fi (2021, unknown): “Traditional approaches use cameras [1], [2], [3], wearable devices and phones [4], [5], [6] or sonar [7], [8], [9] as the sensing module.”
Human Activity Recognition using Inertial, Physiological and Environmental Sensors: A Comprehensive Survey (2020, methodology): “The second most used model was the Long Short-TermMemory (LSTM) model, which was used in 17 papers [7], [83], [89], [102], [107], [112], [125], [130], [137], [139], [152], [153], [172],”