Designing Neural Networks Using Genetic Algorithms with Graph Generation System

By Hiroaki Kitano. 1990

Subfields: Neural Architecture Search & AutoML, Evolutionary Computation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Neural Architecture Search & AutoML

In Evolutionary Computation

Citations in the corpus

Neural Architecture Optimization (2018, background): “l resources (less than 10 GPU hours) for both tasks. 1 Introduction Automatic design of neural network architecture without human intervention has been the interests of the community from decades ago [12, 22] to very recent [47, 48, 27, 39, 7]. The latest algorithms for automatic architecture design”
Deep learning in neural networks: An overview (2014, background): “irectly (Sec. 6.6), one can sometimes greatly reduce the search space by evolving compact encodings of NNs, e.g., through Lindenmeyer Systems (Lindenmayer, 1968; Jacob et al., 1994), graph rewriting (Kitano, 1990), Cellular Encoding (Gruau et al., 1996), and HyperNEAT (D’Ambrosio and Stanley, 2007; ”
BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Search (2019, background): “NAS has been studied since at least the 1990s and has gained significant attention in the past few years (Kitano 1990; Stanley and Miikkulainen 2002; Zoph and Le 2017).”
Bayesian Learning of Neural Network Architectures (2019, methodology): “…network architecture search has long been a topic of research and diverse methods such as evolutionary algorithms [Todd, 1988, Miller et al., 1989, Kitano, 1990], reinforcement learning [Zoph and Le, 2016] or Bayesian optimisation [Bergstra et al., 2013, Mendoza et al., 2016] have been applied.”
Neural Architecture Search with Bayesian Optimisation and Optimal Transport (2018, methodology): “1 ∗ ||E| − 50|) + frac_sigmoid f3 = f0 + frac_sigmoid D Additional Discussion on Related Work Historically, evolutionary (genetic) algorithms (EA) have been the most common method used for designing architectures [8, 18, 23, 27, 33, 38, 48].”
On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, methodology): “, through Lindenmeyer Systems [115], graph rewriting [127], Cellular Encoding [83], HyperNEAT [268], and other techniques [245, Sec.”
Stochastic Optimization: a Review (2002, background): “Examples of comparisons between binary coding and real-valued or even multiple-character coding include the Kitano (1990) many-character representation for graph-generation grammars, the Meyer & Packard (1992) real-valued representation for condition sets, the Montana & Davies (1989) real-valued…”
Meta learning evolutionary artificial neural networks (2004, background): “various segments or areas. Each segment or area will define a set of neurons, their spatial arrangement and their efferent connectivity. Several high level coding schemes like graph generation system [49], Symbiotic Adaptive Neuro-Evolution (SANE) [62] [65], marker based genetic coding [36], L-syste”
Hierarchical Representations for Efficient Architecture Search (2017, background): “The beneﬁts of indirect encoding schemes, such as multi-scale representations, have historically been discussed in Gruau et al. (1994); Kitano (1990); Stanley (2007); Stanley et al. (2009).”
Designing application-specific neural networks using the structured genetic algorithm (1992, methodology): “But in dealing with the larger network architectures (currently under investigation) it may be useful to use a graph grammatical encoding as mentioned by (Kitano, 1990a) for determining optimal connectivity.”
Learning to Adapt to Changing Environments in Evolving Neural Networks (1996, unknown): “Cf. Belew et al., 1989; Kitano, 1990 .) This is Generation 0 (G0).”
Evolving 3D Morphology and Behavior by Competition (1994, methodology): “This approach is related to Lsystems, graftal grammars, and object instancing techniques [8, 11 ,13,15,23].”
Evolving artificial neural networks (1999, methodology): “They have reimplemented Kitano’s system and discovered that the performance difference between the direct and indirect encoding schemes was not caused by the encoding scheme itself, but by how sparsely connected the initial ANN architectures were in the initial population [151].”
The road to everywhere : Evolution, complexity and progress in natural and artificial systems (2008, methodology): “Kitano [96] was first to use L-systems for evolving neural network (a critical description can be found in [161]).”
Robust non-linear control through neuroevolution (2003, background): “Some simply provide a coarse description such as delineating a neuron’s receptive field (Mandischer 1993) or connective density (Harp et al. 1989), while others are more algorithmic providing growth rules in the form of graph generating grammars (Kitano 1990; Voigt et al. 1993).”
The evolvability of artificial neural net-works for robot control (2002, unknown): “, 1998), and developmental grammar rules (Kitano, 1990).”
Parallel Problem Solving from Nature - PPSN VIII (2004, methodology): “In [12, 14] Harik and Lobo introduce a parameter-less GA (PLGA) which evolves a number of populations of different sizes simultaneously.”