Learning Unambiguous Reduced Sequence Descriptions

By J. Urgen Schmidhuber. 1991

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Selective Experience Replay for Lifelong Learning (2018, methodology): “…a particular transition This idea is similar to the idea of uncertainty sampling in active learning (Settles 2010), identifying unexpected inputs in history compression (Schmidhuber 1993), and it has been effectively used as the foundation for a prioritized sampling strategy (Schaul et al. 2016).”
Learning long-term dependencies in NARX recurrent neural networks (1996, methodology): “As an alternative to using di erent learning algorithms, one suggestion has been to alter the input data so that it represents a reduced description that makes global features more explicit and more readily detectable [21, 29, 28].”
Making Working Memory Work: A Computational Model of Learning in the Prefrontal Cortex and Basal Ganglia (2006, methodology): “After presenting the PBWM model and its computational, biological, and cognitive bases, we compare its performance with that of several other standard temporal learning models including LSTM, a simple recurrent network (SRN, Elman, 1990; Jordan, 1986), and real-time recurrent backpropagation learnin”
CHILD: A First Step Towards Continual Learning (1997, background): “Another constructive bottom-up approach is Dawkins' " hierarchy of decisions " (Dawkins, 1976), similar to Schmidhuber's " history compression " (Schmidhuber, 1992), where an item in a sequence that reliably predicts the next several items is used to represent the predicted items in a reduced…”
Long Short-Term Memory (1997, unknown): “BPTT: Same architecture as the one trained by RTRL. CH: Both net architectures like RTRL’s, but one has an additional output for predicting the hidden unit of the other one (see Schmidhuber, 1992b, for details).”
Sequential neural text compression (1996, background): “[11]), and variants of the \principle of history compression" [7] [5].”
Learning Sequential Tasks by Incrementally Adding Higher Orders (1992, methodology): “This method concentrating on unexpected events is similar to the "hierarchy of decisions" of Dawkins [3], and the "history compression" of Schmidhuber [13].”
Incremental Slow Feature Analysis: Adaptive Low-Complexity Slow Feature Updating from High-Dimensional Input Streams (2012, methodology): “…1994; Lee & Seung, 1999; Kohonen, 2001; Hinton, 2002), which ignore dynamics, and other UL systems that both learn and apply features to sequences (Schmidhuber, 1992a, 1992b, 1992c; Lindstädt, 1993; Klapper-Rybicka, Schraudolph, & Schmidhuber, 2001; Jenkins & Matarić, 2004; Lee, Largman, Pham,…”
Neural net architectures for temporal sequence processing (2007, methodology): “…net architecture trained with back propagation through time (or BPTT; Rumelhart, Hinton, & Williams, 1986) or real-time recurrent learning (or RTRL; Robinson & Fallside, 1987; Schmidhuber, 1992; Williams & Zipser, 1989) is also an adaptive memory model in that the training adjusts w of Equation 2.”
Continual learning in reinforcement environments (1995, unknown): “Another constructive , bottom - up approach is the \hierarchy of decisions " of Dawkins 18 ] , similar to \history compression , " recently described and implemented by Schmidhuber 96 ] .”