Modeling systems with internal state using evolino

By Daan Wierstra, Faustino J. Gomez, J. Urgen Schmidhuber. 2005

65 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Cognitive Neuroscience, Evolutionary Computation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

In Evolutionary Computation

Citations in the corpus

MaskConnect: Connectivity Learning by Gradient Descent (2018, positive): “Evolutionary search has been proposed as an interesting framework to learn both the structure as well as the connections in a neural network [16,17,18,19,20,21,22,23,24].”
Connectivity Learning in Multi-Branch Networks (2017, background): “ 2015) rather than architecture or connectivity learning. Evolutionary search has been proposed as an interesting framework to learn both the structure as well as the connections in a neural network (Wierstra et al., 2005; Floreano et al., 2008; Real et al., 2017). Architecture search has also been ”
SMASH: One-Shot Model Architecture Search through HyperNetworks (2017, background): “Evolutionary techniques [10, 37, 38, 40] offer a ﬂexible approach for discovering variegated models from trivial initial conditions, but often struggle to scale to deep neural nets where the search space is vast, even with enormous compute power [26].”
DeepArchitect: Automatically Designing and Training Deep Architectures (2017, neutral): “[29], Floreano et al .”
Learning Transferable Architectures for Scalable Image Recognition (2017, background): “roaches in designing architectures such as Neural Fabrics [48], DiffRNN [41], MetaQNN [3] and DeepArchitect [43]. A more ﬂexible class of methods for designing architecture is evolutionary algorithms [65, 16, 57, 30, 46, 42, 67], yet they have not had as much success at large scale. Xie and Yuille [”
Supervised Sequence Labelling with Recurrent Neural Networks (2012, background): “Recently, non gradient-based training methods of LSTM have also been considered (Wierstra et al., 2005; Schmidhuber et al., 2007), but they are outside the scope of this book.”
Dynamic Factor Graphs for Time Series Modeling (2009, methodology): “DFGs outperformed LSTMs on that task since the multi-step iterated (closed-loop) prediction of DFG did not decrease in SNR even after thousands of iterations, contrary to [21] where a reduction in SNR was already observed after around 700 iterations.”
Neural Architecture Search with Reinforcement Learning (2016, background): “In the last part of the section, we will focus on generating recurrent architectures, which is another key contribution of our paper.”
A review on the long short-term memory model (2020, background): “Alternatively, one can train the model with evolino (Schmidhuber et al. 2007; Wierstra et al. 2005).”
Learning to Reason: End-to-End Module Networks for Visual Question Answering (2017, methodology): “Past work includes techniques for optimizing over the space of architectures using evolutionary algorithms [23, 8], Bayesian methods [6], and reinforcement learning [28].”