A local learning algorithm for dynamic feedforward and recurrent networks

By J. Urgen Schmidhuber. 1990

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Deep learning in neural networks: An overview (2014, methodology): “10, 1991), RL economies in RNNs with WTA units and local learning rules (Schmidhuber, 1989b), and other methods (e.”
First Experiments with PowerPlay (2012, methodology): “Inspired by a previous architecture [15], neurons other than the inputs and outputs in our RNN are arranged in winner-take-all subsets (WITAS) of n witas neurons each ( n witas = 4 was used for this experiment).”
On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, methodology): “But we can implement M as a self-modularizing, computation cost-minimizing, winner-take-all RNN [221, 242, 267].”
One Big Net For Everything (2018, methodology): “To avoid forgetting previous skills, instead of replaying all previous traces of still relevant trials (the simplest option to achieve the P OWER P LAY criterion [75]), one can also implement ONE as a self-modularizing, computation cost-minimizing, winner-take-all RNN [53, 74, 86].”
Learning to Understand Phrases by Embedding the Dictionary (2015, methodology): “For this purpose we use a recurrent neural network (RNN) (Schmidhuber, 1989) with long-short-term memory (LSTM) (Hochreiter and Schmidhuber, 1997).”
My First Deep Learning System of 1991 + Deep Learning Timeline 1962-2013 (2013, methodology): “…(TL 1.12-1.13) are GPU-based [18] multi-column [22] combinations of two ancient concepts: Backpropagation (TL 1.2) applied [55] to Neocognitron-like convolutional architectures (TL 1.3) (with max-pooling layers [72, 71, 77] instead of alternative [29, 30, 31, 78] local winner-take-all methods).”
Learning Unambiguous Reduced Sequence Descriptions (1991, background): “Since only unpredictable inputs convey new information, ignore all predictable inputs but let all unexpected inputs (plus information about the time step at which they occurred) become inputs to a higher-level network of the same kind (working on a slower, self-adjusting time scale).”
Long Short-Term Memory (1997, unknown): “Our simple 3 Following Schmidhuber (1989), we say that a recurrent net algorithm is local in space if the update complexity per time step and weight does not depend on network size.”
Understanding Locally Competitive Networks (2014, background): “Related activation techniques have been studied in the past decades, including recurrent networks with locally competitive units [8].”
Learning to Control Fast-Weight Memories: An Alternative to Dynamic Recurrent Networks (1992, methodology): “Previous approaches to solving this problem employed dynamic recurrent nets (e.g. [2], [12], [1], [13], [6]).”
Learning Algorithms for Networks with Internal and External Feedback (1990, background): “This makes it biologically more plausible than other algorithms (Schmidhuber, 1989a) (Schmidhuber, 1989b).”
Compete to Compute (2013, background): “Nonetheless, networks employing local competition have existed since the late 80s [21], and, along with [22], serve as a primary inspiration for the present work.”
Networks adjusting networks (1990, methodology): “A few methods are designed for reinforcement learning with recurrent neural networks, including `extended REINFORCE' algorithms [20], the `Neural Bucket Brigade Algorithm' [9], and model-building recurrent neural controllers based on system identi cation (e.”
Formal Theory of Creativity, Fun, and Intrinsic Motivation (1990–2010) (2010, background): “[30], [65], [66], [120] may be viewed as translating rare ex-”