2007 Special Issue: Optimization and applications of echo state networks with leaky- integrator neurons

By Herbert Jaeger, Mantas Lukosevicius, Dan Popovici, Udo Siewert. 2007

954 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Electrical and Electronic Engineering, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Electrical and Electronic Engineering

In Model Architectures

Citations in the corpus

Learning to Execute (2014, methodology): “…to use Recurrent Neural Networks with Long Short Term Memory units (Hochreiter & Schmidhuber, 1997), although there are many other RNN variants that perform well on tasks with long term dependencies (Cho et al., 2014; Jaeger et al., 2007; Koutn´ık et al., 2014; Martens, 2010; Bengio et al., 2013).”
On the difficulty of training recurrent neural networks (2012, methodology): “An extension to the classical model is represented by leaky integration units (Jaeger et al. , 2007), where While these units can be used to solve the standard benchmark proposed by Hochreiter and Schmidhu-ber (1997) for learning long term dependencies (see (Jaeger, 2012)), they are more suitable…”
Recurrent Neural Network Regularization (2014, background): “There have been a number of architectural variants of the RNN that perform better on problems with long term dependencies (Hochreiter & Schmidhuber, 1997; Graves et al., 2009; Cho et al., 2014; Jaeger et al., 2007; Koutn´ık et al., 2014; Sundermeyer et al., 2012).”
Re-visiting the echo state property (2012, background): “For dealing with slowly and continuously changing systems, a continuous version of ESN was introduced in [8] and investigated in more detail in [13].”
Interference Management for Cellular-Connected UAVs: A Deep Reinforcement Learning Approach (2018, background): “In particular, we consider leaky integrator reservoir units [29] for updating the state transition functions x where ω ( n ) j ∈ [0 , 1] is the leaking parameter at layer n for UAV j which relates to the speed of the reservoir dynamics in response to the input, with larger values of ω resulting in…”
Controlling Recurrent Neural Networks by Conceptors (2014, methodology): “The linear classifier that serves as a baseline comparison was designed in essentially the same way as the Echo State Networks based classifiers which in [57] yielded zero test misclassifications (when combining 1,000 such classifiers made from 4-unit networks) and 2 test misclassifications (when a ”
A Practical Guide to Applying Echo State Networks (2012, background): “The leaking rate α can even be adapted online to deal with time wrapping of the signals [27,12].”
Stepping forward through echoes of the past : forecasting with Echo State Networks (2007, methodology): “Jaeger has found that ESNs can classify stochastic (speech) time series very well when a large number of very small ESNs are combined in a voting collective [5].”
Selection of cortical dynamics for motor behaviour by the basal ganglia (2015, background): “In all simulations lateral connection weights Wu were generated randomly and normalized following the constraints of leaky echo state reservoirs (Jaeger et al. 2007), with a further transfomation to improve the richness of the dynamics (see Appendix 1).”
Acoustic Modeling With Hierarchical Reservoirs (2013, background): “In that case, the reservoir neurons are called Leaky Integrator Neurons (LIN, [47]) and is the leak rate.”