Evolving neural networks in compressed weight space

By Jan Koutník, Faustino J. Gomez, J. Urgen Schmidhuber. 2010

95 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Control and Systems Engineering, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Control and Systems Engineering

In Representation & Self-Supervised Learning

Citations in the corpus

HyperNetworks (2016, methodology): “First, it can be argued that Discrete Cosine Transform used in Compressed Weight Search may be too simple and using the DCT prior may not be suitable for many problems.”
Self-Delimiting Neural Networks (2012, background): “Yet additional variants of adaptive universal search for low-complexity networks search in compressed network space [36, 35, 18].”
Evolving large-scale neural networks for vision-based TORCS (2013, methodology): “For problems exhibiting a highdegree of redundancy, this “compressed” approach can result in an order of magnitude fewer free parameters and significant speedup [7].”
Evolving deep unsupervised convolutional networks for vision-based reinforcement learning (2014, background): “Most approaches to scaling neuroevolution to tasks that require large networks, such as those processing video input, have focused on indirect encodings where relatively small neural network descriptions are transformed via a complex mapping into networks of arbitrary size [4, 7, 10, 12, 14, 21].”
Online Evolution of Deep Convolutional Network for Vision-Based Reinforcement Learning (2014, background): “Most approaches to scaling neuroevolution to tasks that require large networks, such as those processing video input, have focused on indirect encodings where relatively small neural network descriptions are transformed via a complex mapping into networks of arbitrary size [4, 7, 10,13,15,23].”
Intrinsically motivated neuroevolution for vision-based reinforcement learning (2011, background): “Most approaches to scaling these methods have focused on indirect encodings where relatively small network descriptions are transformed via a complex mapping into networks of arbitrary size [3,8,12].”
Cartesian genetic programming encoded artificial neural networks: a comparison using three benchmarks (2013, methodology): “The Ball Throwing experiment is taken from [21] where it was used as a comparison between CoSyNE and Compressed CoSyNE training algorithms.”
Evolving large-scale neural networks for vision-based reinforcement learning (2013, background): “For problems exhibiting a highdegree of redundancy, this “compressed” approach can result in an two orders of magnitude fewer free parameters and significant speedup [9].”
Text Generation with Exemplar-based Adaptive Decoding (2019, unknown): “The idea of using a smaller network to generate weights for a larger one dues back to Stanley et al. (2009) and Koutnik et al. (2010), mainly under the evolution computing context.”
Principled Weight Initialization for Hypernetworks (2020, background): “Other non-hypernetwork methods of neural network generation (Stanley et al., 2009; Koutnik et al., 2010) can also be improved by considering whether their generated weights result in exploding activations and how to avoid that if so.”
Autonomous reinforcement learning with experience replay. (2013, background): “Because the original state- and action space of an octopus arm has huge number of dimensions, it often serves as a test bed for solutions to the problem of dimensionality in RL (Koutnik et al., 2010; Woolley & Stanley, 2010).”