Online Learning with Random Representations

By Richard S. Sutton, Steven D. Whitehead. 1993

Subfields: Representation & Self-Supervised Learning, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

In Statistical Learning Theory

Citations in the corpus

Toward Understanding Catastrophic Forgetting in Continual Learning (2019, background): “Continual learning (or life-long learning) [32, 37, 44] is the ability of a machine learning model to continuously learn from a stream of data, which could possibly be non-iid or come from different but related tasks.”
Variational Continual Learning (2017, background): “…incremental learning) is a very general form of online learning in which data continuously arrive in a possibly non i.i.d. way, tasks may change over time (e.g. new classes may be discovered), and entirely new tasks can emerge (Schlimmer and Fisher, 1986; Sutton and Whitehead, 1993; Ring, 1997).”
Continual Learning with Adaptive Weights (CLAW) (2019, background): “Data belonging to different tasks could potentially be non i.i.d. (Schlimmer & Fisher, 1986; Sutton & Whitehead, 1993; Ring, 1997; Schmidhuber, 2013; Nguyen et al., 2018; Schmidhuber, 2018).”
Reinforcement Learning with Replacing Eligibility Traces (2005, background): “…clear howto extend the replace-trace idea to other kinds of function approximators, such asback-propagation networks or nearest-neighbor methods, Sutton and Whitehead(1993) and others have argued that feature-based function approximators are ac-tually preferable for online reinforcement…”
Overcoming Catastrophic Forgetting by Bayesian Generative Regularization (2019, background): “The labelled data of an existing task can be depleted when new tasks emerge (Sutton et al., 2014; Kirk-patrick et al., 2017).”
Online Extreme Evolutionary Learning Machines (2014, unknown): “The nonlinearities in the hidden layer are achieved by means of Linear Threshold Units (LTUs) adopted from (Sutton and Whitehead, 1993).”
Incremental Off-policy Reinforcement Learning Algorithms (2017, background): “Sutton and Whitehead (1993) coined the term strict incrementality for this computational goal.”
Learning Rate Adaptation in Stochastic Gradient Descent (2001, methodology): “Moreover, for the comparisons we have also tested the on–line Back Propagation (On–line BP) [18], the batch Back Propagation (Batch BP) [18], and the batch adaptive BP with adaptive momentum (Batch ABP) [25].”
Reinforcement learning with replacing eligibility traces (2004, background): “Although it is not yet clear how to extend the replace-trace idea to other kinds of function approximators, such as back-propagation networks or nearest-neighbor methods, Sutton and Whitehead (1993) and others have argued that feature-based function approximators are actually preferable for online”
Loss of plasticity in deep continual learning (2024, unknown): “The threshold θ i is set to be equal to ( m + 1) × β − S i , in which β ∈ [0, 1] and S i is the number of input weights with negative value 63 .”
Partitioned Variational Inference: A unified framework encompassing federated and continual learning (2018, background): “…learning or life-long learning or incremental learning) is the ability to learn continually and adapt quickly to new experiences without catastrophically forgetting previously seen experiences [Schlimmer and Fisher, 1986, McCloskey and Cohen, 1989, Sutton and Whitehead, 1993, Ratcliff, 1990].”
Overcoming Catastrophic Forgetting by Generative Regularization (2019, background): “Critically, the sequence of tasks may not be explicitly labelled, tasks may change over time, any individual task may not recur for long time intervals and entirely new tasks can emerge (Sutton et al., 2014; Kirkpatrick et al., 2017).”
Model-Free reinforcement learning with continuous action in practice (2012, unknown): “Second, the algorithm should be strictly incremental in that its per-time-step computational requirements do not increase with time [8].”
A Study of Reinforcement Learning in the Continuous Case by the Means of Viscosity Solutions (2000, methodology): “…with neural networks (see for example (Barto, Sutton, & Anderson, 1983), (Barto, 1990), (Gullapalli, 1992), (Williams, 1992), (Lin, 1993), (Sutton & Whitehead, 1993), (Harmon, Baird, & Klopf, 1996), and (Bertsekas & Tsitsiklis, 1996)), fuzzy sets (see (Now e, 1995), (Glorennec & Jou e,…”