The Cascade-Correlation Learning Architecture

By Scott E. Fahlman, Christian Lebiere. 1989

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Model Architectures

Citations in the corpus

Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation (2016, background): “Motivated by the idea of modeling differences between an intermediate layer’s output and the targets, which has shown to work well for many projects in the past [16, 21, 40], we introduce residual connections among the LSTM layers in a stack (see Figure 2).”
Deep Networks with Stochastic Depth (2016, methodology): “Earlier works adopted greedy layer-wise training or better initialization schemes to alleviate the vanishing gradients and diminishing feature reuse problems [12, 17, 18].”
The Graph Neural Network Model (2009, neutral): “Thus, second-order learning algorithms [62], pruning [63], and growing learning algorithms [64]–[66] designed for static networks cannot be directly applied to GNNs.”
Progressive Neural Networks (2016, methodology): “The cascade-correlation architecture was designed to eliminate forgetting while incrementally adding and refining feature extractors [6].”
Regularized Evolution for Image Classifier Architecture Search (2018, neutral): “siﬁer architecture search, including successful RL studies [2,6,28, 51–53] and evolutionary studies like those mentioned in the Introduction. Other methods have also been applied: cascade-correlation [16], boosting [10], hill-climbing [14], MCTS [33], SMBO [28,30], and random search [4], and grid se”
AutoML-Zero: Evolving Machine Learning Algorithms From Scratch (2020, methodology): “The length and difficulty of ML research prompted a new field, named AutoML, that aims to automate such progress by spending machine compute time instead of human research time (Fahlman & Lebiere, 1990; Hutter et al., 2011; Finn et al., 2017).”
Evolutionary-Neural Hybrid Agents for Architecture Search (2018, neutral): “…deep RL and evolution, different approaches have been applied to architecture search and hyperparameter optimization: bayesian optimization [15], cascade-correlation [14], boosting [9], deep-learning based tree search [30], hill-climbing [13], model-based approaches [21], and random search [4].”
Net2Net: Accelerating Learning via Knowledge Transfer (2015, methodology): “In this section, we describe our new Net2Net operations and how we applied them on real deep neural nets.”
Rethinking eliminative connectionism. (1998, methodology): “These weights are subsequently adjusted through training with the back-propagation algorithm or one of its variants (e.g., Fahlman & Lebiere, 1990).”
An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solution (2018, background): “Finally, we note that challenges in learning coordinate transformations are not unknown in machine learning, as learning a Cartesian-to-polar coordinate transform forms the basis of the classic twospirals classification problem [4].”
Diffusion-based neuromodulation can eliminate catastrophic forgetting in simple neural networks (2017, background): “In such a situation, any change to a group of nodes and connections to learn new information would cause forgetting because those nodes and connections most likely encoded for something else [14,15].”
Deep Growing Learning (2017, unknown): “To address the difficulty of training deep multi-layer neural networks, an approach [7] based on constructively adding layers in a supervised fashion has been explored with some success.”
Breast Cancer Diagnosis and Prognosis Via Linear Programming (1995, methodology): “In order to add more predictive power to the RSA method, we are implementing extensions similar to Wolpert’s Stacked Generalization [24] and Fahlman’s Cascade Correlation [7].”
Online Incremental Feature Learning with Denoising Autoencoders (2012, methodology): “In addition to boosting, our model is also related to cascade correlation [12].”