Novelty of behaviour as a basis for the neuro-evolution of operant reward learning

By Andrea Soltoggio, Ben Jones. 2009

24 citations recorded by the source corpus.

An agent that deviates from a usual or previous course of action can be said to display novel or varying behaviour. Novelty of behaviour can be seen as the result of real or apparent randomness in decision making, which prevents an agent from repeating exactly past choices. In this paper, novelty of behaviour is considered as an evolutionary precursor of the exploring skill in reward learning, and conservative behaviour as the precursor of exploitation. Novelty of behaviour in neural control is hypothesised to be an important factor in the neuro-evolution of operant reward learning. Agents capable of varying behaviour, as opposed to conservative, when exposed to reward stimuli appear to acquire on a faster evolutionary scale the meaning and use of such reward information. The hypothesis is validated by comparing the performance during evolution in two environments that either favour or are neutral to novelty. Following these findings, we suggest that neuro-evolution of operant reward learning is fostered by environments where behavioural novelty is intrinsically beneficial, i.e. where varying or exploring behaviour is associated with low risk.

Subfields: Cognitive Neuroscience

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

Citations in the corpus

Born to Learn: the Inspiration, Progress, and Future of Evolved Plastic Artificial Neural Networks (2017, methodology): “With the aim of validating the same hypothesis, Soltoggio and Jones (2009) devised a simple EPANN experiment in which exploring was more advantageous than exploiting when reward information was not used, e.g. , before learning evolves; to do this, the reward at a particular location depleted itself…”
Artificial Evolution of Plastic Neural Networks: A Few Key Concepts (2014, methodology): “For instance, the discrete T-maze exper-iment [17,18,21–23] can be described as a special case of a Skinner box.”
Abandoning Objectives: Evolution Through the Search for Novelty Alone (2011, background): “Two independent works have also demonstrated that behavioral novelty is useful in evolving adaptive neural networks (i.e., neural networks that learn during their lifetimes; Risi et al., 2009; Soltoggio and Jones, 2009).”
Revising the evolutionary computation abstraction: minimal criteria novelty search (2010, unknown): “ANNs that learn during their lifetimes) [25, 29].”
Using a map-based encoding to evolve plastic neural networks (2011, methodology): “These results are similar to the ones obtained by Risi [29] and Soltoggio [7].”
Overcoming deception in evolution of cognitive behaviors (2014, methodology): “The Standard T-Maze map (figure 2a) is similar to the classic T-Maze imported from experiments in animal cognition [14], and has been previously explored in previous ER studies [13, 17, 21].”
Quality Diversity: A New Frontier for Evolutionary Computation (2016, background): “, and Stanley, K. O. (2015). “Confronting the challenge of quality diversity,” in Proceedings of the 17th Annual Conference on Genetic and Evolutionary Computation (GECCO ‘15) (New York, NY: ACM). Risi, S., Hughes, C., and Stanley, K. (2011). Evolving plastic neural networks with novelty search. Ada”
Real-time Hebbian Learning from Autoencoder Features for Control Tasks (2014, background): “As a medium for adaptation and learning, neural plasticity has long captivated artificial life and related fields (Baxter, 1992; Floreano and Urzelai, 2000; Niv et al., 2002; Soltoggio et al., 2008, 2007; Soltoggio and Jones, 2009; Soltoggio and Stanley, 2012; Risi and Stanley, 2010; Risi et al., 20”
On the Relationships between Generative Encodings, Regularity, and Learning Abilities when Evolving Plastic Artificial Neural Networks (2013, background): “To mitigate premature convergence, we add to the fitness objective a behavioral novelty objective [4,22,23,48,49] that rewards individuals that do something that has not been done before.”