LSTM-CNN Architecture for Human Activity Recognition

By Kun Xia, Jianguang Huang, Hanyu Wang. IEEE Access, 2020

675 citations recorded by the source corpus.

In the past years, traditional pattern recognition methods have made great progress. However, these methods rely heavily on manual feature extraction, which may hinder the generalization model performance. With the increasing popularity and success of deep learning methods, using these techniques to recognize human actions in mobile and wearable computing scenarios has attracted widespread attention. In this paper, a deep neural network that combines convolutional layers with long short-term memory (LSTM) was proposed. This model could extract activity features automatically and classify them with a few model parameters. LSTM is a variant of the recurrent neural network (RNN), which is more suitable for processing temporal sequences. In the proposed architecture, the raw data collected by mobile sensors was fed into a two-layer LSTM followed by convolutional layers. In addition, a global average pooling layer (GAP) was applied to replace the fully connected layer after convolution for reducing model parameters. Moreover, a batch normalization layer (BN) was added after the GAP layer to speed up the convergence, and obvious results were achieved. The model performance was evaluated on three public datasets (UCI, WISDM, and OPPORTUNITY). Finally, the overall accuracy of the model in the UCI-HAR dataset is 95.78%, in the WISDM dataset is 95.85%, and in the OPPORTUNITY dataset is 92.63%. The results show that the proposed model has higher robustness and better activity detection capability than some of the reported results. It can not only adaptively extract activity features, but also has fewer parameters and higher accuracy.

Subfields: Biomedical Engineering, Computer Networks and Communications, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/29a583f4bb6c2794c2f4400beb29d64aca3f210a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

In Computer Networks and Communications

In Computer Vision and Pattern Recognition

Citations in the corpus

Deep Learning in Human Activity Recognition with Wearable Sensors: A Review on Advances (2021, background): “Similarly, other researchers [8, 82, 94, 95, 197, 245, 301, 302], [238] have also developed the CNN-LSTM model in varios application scenarios by taking advantage of the feature extraction ability of CNN and the time-series data reasoning ability of LSTM.”
Human Activity Recognition With Smartphone and Wearable Sensors Using Deep Learning Techniques: A Review (2021, methodology): “Xia et al. [55] have proposed a deep hybrid model by integrating LSTM models connected with Convolutional Layers and Global Pooling layers (GAP).”
Deep ConvLSTM With Self-Attention for Human Activity Decoding Using Wearable Sensors (2020, background): “Together with CNN and LSTM, we show that the self-attention mechanism leads to a statistically significant improvement in accuracy for six benchmark datasets.”
A review of machine learning in processing remote sensing data for mineral exploration (2021, background): “nation for image datasets with temporal features such as realtime video processing (Xia et al., 2020).”