Learning long-term dependencies is not as difficult with NARX recurrent neural networks

By Tsungnan Lin, Bill G. Horne, Peter Tino, C. Lee Giles. 1995

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Deep learning in neural networks: An overview (2014, background): “Notably, an NN with internal delay lines won the Santa Fe time-series competition on chaotic intensity pulsations of an NH3 laser (Wan, 1994; Weigend&Gershenfeld, 1993).”
Architectural Complexity Measures of Recurrent Neural Networks (2016, background): “Skip connections crossing different timescales have also been studied by Lin et al. (1996); El Hihi & Bengio (1996); Sutskever & Hinton (2010); Koutnik et al. (2014).”
Ordered Neurons: Integrating Tree Structures into Recurrent Neural Networks (2018, methodology): “There has been precedence for such models: El Hihi & Bengio (1996); Schmidhuber (1991); Lin et al. (1998) describe models that model data at different, pre-determined time-scales.”
Computational capabilities of recurrent NARX neural networks (1997, background): “We have also shown that NARX networks perform better on problems involving long-term dependencies [20].”
Guessing can Outperform Many Long Time Lag Algorithms (1996, background): “Quite a few papers use Bengio et al.'s \2-sequence problem" (and \latch problem") toshow the proposed algorithms's superiority, e.g., Bengio et al. (1994), Bengio and Frasconi (1994),El Hihi and Bengio (1995), Lin et al. (1995).”
Learning Deep Architectures for AI (2007, background): “There might also be a link between this difficulty in exploiting the gradient in deep networks and the difficulty in training recurrent neural networks through long sequences, analyzed in [22, 81, 119].”
Bridging Long Time Lags by Weight Guessing and \Long Short Term Memory" (1996, background): “See, e.g., Bengio et al., El Hihi and Bengio, and others [3, 1, 6, 15].”