Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication

By Herbert Jaeger, Harald Haas. 2004

3,739 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Electrical and Electronic Engineering, Signal Processing, Time-Series & Forecasting

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yoshua Bengio
2. Sergey Levine
3. J. Urgen Schmidhuber
4. Quoc V. Le
5. Ruslan Salakhutdinov
6. Pieter Abbeel
7. Mohit Bansal
8. Jianfeng Gao

In Electrical and Electronic Engineering

1. J. Urgen Schmidhuber
2. Herbert Jaeger
3. Walid Saad
4. Merouane Debbah
5. Andrea Soltoggio
6. Yoshua Bengio
7. Mingzhe Chen
8. Mantas Lukosevicius

In Signal Processing

1. Yoshua Bengio
2. J. Urgen Schmidhuber
3. Li Deng
4. Dong Yu
5. Ilya Sutskever
6. Geoffrey E. Hinton
7. Jinyu Li
8. Quoc V. Le

In Time-Series & Forecasting

1. Konstantinos Nikolopoulos
2. J. Urgen Schmidhuber
3. Sven F. Crone
4. Mantas Lukosevicius
5. Stefan L. Frank
6. Miquel Perelló Nieto
7. Srdjan Ostojic
8. Yoshua Bengio

Citations in the corpus

• On the difficulty of training recurrent neural networks (2012, background): “Echo State Networks (Jaeger and Haas, 2004) avoid the exploding and vanishing gradients problem by not learning Wrec and Win.”
• Pretrained Transformers as Universal Computation Engines (2021, methodology): “2019) and echo state networks (Jaeger 2001; Jaeger and Haas 2004), a random recurrent network is frozen and only the output readout layer is trained.”
• Weight Agnostic Neural Networks (2019, background): “ainting and style transfer. Schmidhuber et al. [103] have shown that a randomly-initialized LSTM [49] with a learned linear output layer can predict time series where traditional reservoir-based RNNs [51, 99] fail. More recent developments in self-attention [121] and capsule [100] networks expand th”
• Deep learning in neural networks: An overview (2014, methodology): “In fact, some popular RNN algorithms restricted credit assignment to a single step backwards (Elman, 1990; Jordan, 1986, 1997), also inmore recent studies (Jaeger, 2001, 2004; Maass et al., 2002).”
• Born to Learn: the Inspiration, Progress, and Future of Evolved Plastic Artificial Neural Networks (2017, background): “(2007), showing that the evolved RNNs were more compact and resulted in better learning than randomly initialized echo state networks (Jaeger and Haas, 2004).”
• Deep Fried Convnets (2014, background): “The intriguing idea of constructing neural networks with random weights has been reasonably explored in the neural networks field [34, 19].”
• On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, positive): “Those early CM systems, however, did not yet use powerful RNNs such as LSTM.”
• Capacity and Trainability in Recurrent Neural Networks (2016, background): “Previous work on short term memory traces explores the tradeoffs between memory fidelity and duration, for the case that a new input is presented to the RNN at every time step (Jaeger & Haas, 2004; Maass et al., 2002; White et al., 2004; Ganguli et al., 2008; Charles et al., 2014).”
• Self-Delimiting Neural Networks (2012, methodology): “Supervised RNNs can be trained to map sequences of input patterns to desired output sequences by gradient descent and other methods [78, 81, 47, 52, 44, 31, 75].”
• HiPPO: Recurrent Memory with Optimal Polynomial Projections (2020, background): “The deep learning instantiation of this approach is the recurrent neural network (RNN), which is known to suffer from a limited memory horizon [34, 38, 56] (e.”
• Toward an Integration of Deep Learning and Neuroscience (2016, background): “For example, “liquid-” (Maass et al., 2002) or “echo-state machines” (Jaeger and Haas, 2004) are randomly connected recurrent networks that form a basis set (also known as a “reservoir”) of random filters, which can be harnessed for learning with tunable readout weights.”
• Reservoir computing approaches to recurrent neural network training (2009, background): “Demonstrations of RLS are presented in [17, 43].”
• Generating Text with Recurrent Neural Networks (2011, neutral): “Another way to avoid the problems associated with backpropagation through time is the Echo State Network (Jaeger & Haas, 2004) which forgoes learning the recurrent connections altogether and only trains the non-recurrent output weights.”
• Generating Coherent Patterns of Activity from Chaotic Neural Networks (2009, neutral): “As stated in the Introduction, Jaeger and Haas (2004) eliminated the problem of delayed effects by clamping feedback during learning.”