Bifurcations of Recurrent Neural Networks in Gradient Descent Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

On the difficulty of training recurrent neural networks (2012, methodology): “We can improve our understanding of the exploding gradients and vanishing gradients problems by employing a dynamical systems perspective, as it was done before in Doya (1993); Bengio et al. (1993).”
PIPPS: Flexible Model-Based Policy Search Robust to the Curse of Chaos (2019, background): “Exploding gradients have been observed in deep learning research for a long time (Doya, 1993; Bengio et al., 1994).”
Neural Networks and Deep Learning (2018, unknown): “Teacher forcing methods are discussed in [105].”
Sequence classification for credit-card fraud detection (2018, background): “Another proposal is based on the assumption that if the model exhibits from the beginning AC CE PT ED M AN US CR IP T the same kind of asymptotic behavior as the one required by the target, then it’s less likely for gradients to explode [Doya, 1993].”
Understanding the exploding gradient problem (2012, background): “Looking at dynamical systems theory for explaining the exploding gradient problem has been done before in Doya (1993); Bengio et al. (1993).”
Beyond Convexity: Stochastic Quasi-Convex Optimization (2015, background): “Ideas spanning gradient-clipping [16], smart initialization [5], and more [13], have shown to improve training in practice.”