A Review of Recurrent Neural Networks: LSTM Cells and Network Architectures

By Yong Yu, Xiaosheng Si, Changhua Hu, Jian-xun Zhang. Neural Computation, 2019

5,068 citations recorded by the source corpus.

Recurrent neural networks (RNNs) have been widely adopted in research areas concerned with sequential data, such as text, audio, and video. However, RNNs consisting of sigma cells or tanh cells are unable to learn the relevant information of input data when the input gap is large. By introducing gate functions into the cell structure, the long short-term memory (LSTM) could handle the problem of long-term dependencies well. Since its introduction, almost all the exciting results based on RNNs have been achieved by the LSTM. The LSTM has become the focus of deep learning. We review the LSTM cell and its variants to explore the learning capacity of the LSTM cell. Furthermore, the LSTM networks are divided into two broad categories: LSTM-dominated networks and integrated LSTM networks. In addition, their various applications are discussed. Finally, future research directions are presented for LSTM networks.

Subfields: Artificial Intelligence, Signal Processing

View Paper: https://www.semanticscholar.org/paper/d58fd97d7a8308a20b159bbaf02011e26c323852

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

Stock price prediction with optimized deep LSTM network with artificial rabbits optimization algorithm (2023, background): “The elements of an LSTM network include (Gers et al., 2000; Yu et al., 2019): Input gate: Controls the flow of data into the memory cell.”
Autonomous Robotic Laparoscopic Surgery for Intestinal Anastomosis (2022, methodology): “Furthermore, once additional staging features are enabled, the tissue landmark detection methods discussed in this article will need to include architectures such as long short-term memory (43) to handle marked changes in the temporal information caused by rapid position and orientation changes in t”
A comprehensive review on ensemble deep learning: Opportunities and challenges (2023, methodology): “Also, deep learning methods contain several types of network architectures for different tasks, such as feeding forward neural networks (Bebis and Georgiopoulos, 1994), convolutional neural networks (Collobert and Weston, 2008), recurrent neural networks (Yu et al., 2019).”
Sequence Classification Restricted Boltzmann Machines With Gated Units (2020, methodology): “Other variants of gated neural network are also used in a wide range of sequence clas-siﬁcation problems, which can be categorized into architecture variants and cell variants [21].”
Feature dimensionality reduction: a review (2022, methodology): “To alleviate this problem, a variant of the RNN network of long shortterm memory (LSTM) is proposed, which uses the cell state with gate structure tomake information flow continuously [145].”
Implicit Deep Learning (2019, background): “In the case of Long Short-Term Memory (LSTM) [55] or gated recurrent units (GRUs) [15], a basic building block involves the product of two input variables after each one passes”
Deep learning in drug discovery: an integrative review and future challenges (2022, methodology): “GCNs have been employed as 3D descriptors instead of SMILES strings in a lot of research, and it’s been discovered that these learned descriptors outperform standard descriptors in prediction tests and are easier to understand (Shin et al. 2019; Ozturk et al. 2018; Yu et al. 2019).”
Spatio-Temporal Graph Neural Networks for Predictive Learning in Urban Computing: A Survey (2023, background): “This type of hybrid neural architecture is generally known as Spatio-Temporal Graph Neural Network (STGNN) .”
Deep learning and medical image processing for coronavirus (COVID-19) pandemic: A survey (2020, neutral): “In the case of a Recurrent Neural Network (RNN), there might be more than one signal which traverses multiple times in a layer, and thus the CAP depth cannot be determined (Yu, Si, Hu, & Zhang, 2019).”