Gate-variants of Gated Recurrent Unit (GRU) neural networks

By Rahul Dey, Fathi M. Salem. 2017

2,106 citations recorded by the source corpus.

The paper evaluates three variants of the Gated Recurrent Unit (GRU) in recurrent neural networks (RNNs) by retaining the structure and systematically reducing parameters in the update and reset gates. We evaluate the three variant GRU models on MNIST and IMDB datasets and show that these GRU-RNN variant models perform as well as the original GRU RNN model while reducing the computational expense. In this comparative study, we simply refer to the three variants as, respectively, GRU1, GRU2, and GRU3 RNNs.

Subfields: Artificial Intelligence, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistical and Nonlinear Physics

Citations in the corpus

Deep Learning Approach for Short-Term Stock Trends Prediction Based on Two-Stream Gated Recurrent Unit Network (2018, background): “Each recurrent unit in the GRU network [30] captures the dependencies of different time scales adaptively.”
KalmanNet: Neural Network Aided Kalman Filtering for Partially Known Dynamics (2021, background): “This often results in substantial overparameterization, as the number of GRU parameters grows quadratically with the number of state variables [68].”
Deep Learning for Time Series Anomaly Detection: A Survey (2022, neutral): “However, GRU has gating units that inflate the information flow inside the unit without having any separate memory unit, unlike LSTM [48].”
Potential, challenges and future directions for deep learning in prognostics and health management applications (2020, methodology): “The method was further improved by bidirectional LSTM (Graves and Schmidhuber, 2005), and Gated Recurrent Unit (GRU) (Heck and Salem, 2017; Dey and Salemt, 2017).”
Multi-head CNN-RNN for multi-time series anomaly detection: An industrial case study (2019, methodology): “On the other hand, ﬁve types of recurrent layers are examined: RNN [28], LSTM [69], GRU [70], Bi-LSTM , and Bi-GRU .”
Using labeled data to evaluate change detectors in a multivariate streaming environment (2009, positive): “Furthermore, since there are many small ice blocks in the river ice images, a learnable upsampling strategy DUpsampling is utilized to replace the commonly used bilinear interpolation to increase the segmentation accuracy, especially for small ice blocks.”
A Stacked GRU-RNN-Based Approach for Predicting Renewable Energy and Electricity Load for Smart Grid Operation (2021, background): “However, the computational cost of GRU-RNN can still be high for long sequential data [23].”
A Survey of Knowledge Graph Reasoning on Graph Types: Static, Dynamic, and Multi-Modal (2022, background): “1) RNN-Based Model: Recurrent Neural Networks (RNNs) [219], [220], [221] are suitable for mining the changes over time, which can be generally divided into three variants, i.e., basic RNN, LSTM [222], and GRU [223].”
A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT (2023, methodology): “This was followed by the development of Long Short-Term Memory (LSTM) [24] and Gated Recurrent Unit (GRU) [25], which leveraged gating mechanism to control memory during training.”