Real-time Hebbian Learning from Autoencoder Features for Control Tasks
By Justin K. Pugh, Andrea Soltoggio, Kenneth O. Stanley. 2014
5 citations recorded by the source corpus.
Neural plasticity and in particular Hebbian learning play an important role in many research areas related to artficial life. By allowing artificial neural networks (ANNs) to adjust their weights in real time, Hebbian ANNs can adapt over their lifetime. However, even as researchers improve and extend Hebbian learning, a fundamental limitation of such systems is that they learn correlations between preexisting static features and network outputs. A Hebbian ANN could in principle achieve significantly more if it could accumulate new features over its lifetime from which to learn correlations. Interestingly, autoencoders, which have recently gained prominence in deep learning, are themselves in effect a kind of feature accumulator that extract meaningful features from their inputs. The insight in this paper is that if an autoencoder is connected to a Hebbian learning layer, then the resulting Realtime Autoencoder-Augmented Hebbian Network (RAAHN) can actually learn new features (with the autoencoder) while simultaneously learning control policies from those new features (with the Hebbian layer) in real time as an agent experiences its environment. In this paper, the RAAHN is shown in a simulated robot maze navigation experiment to enable a controller to learn the perfect navigation strategy significantly more often than several Hebbian-based variant approaches that lack the autoencoder. In the long run, this approach opens up the intriguing possibility of real-time deep learning for control.
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Sebastian Risi
- Andrea Soltoggio
- Kenneth O. Stanley
- Jeff Clune
- Joel Lehman
- Jean-Baptiste Mouret
- Risto Miikkulainen
- Paul Tonelli
Citations in the corpus
- Born to Learn: the Inspiration, Progress, and Future of Evolved Plastic Artificial Neural Networks (2017, background): “…problems with a reward signal that affects plasticity in the network (Lin, 1993; Niv et al., 2002; Soltoggio and Stanley, 2012; Auerbach et al., 2014; Pugh et al., 2014) Neuromodulated plasticity (Fellous and Linster, 1998) is an important mechanism in EPANNs (reviewed later in Section III-E).”