When Networks Disagree: Ensemble Methods for Hybrid Neural Networks

By Michael P. Perrone, Leon N. Cooper. 1992

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learning (2020, background): “Many researchers demonstrate that a good ensemble is one where the ensemble’s members are both accurate and make independent errors (Perrone & Cooper, 1992; Maclin & Opitz, 1999).”
Data Diversification: A Simple Strategy For Neural Machine Translation (2019, methodology): “Ensemble of models Using multiple models to average the predictions and reduce variance is a typical feature of ensemble methods [16].”
No One Representation to Rule Them All: Overlapping Features of Training Methods (2021, background): “Without behavior diversity, ensemble beneﬁts are limited to reducing noise, since models make correlated errors (Perrone & Cooper, 1992; Opitz & Maclin, 1999).”
Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning (2020, background): “we prove that self-distillation can also be viewed as implicitly combining ensemble and knowledge distillation arXiv:2012.09816v1 [cs.LG] 17 Dec 2020 to improve test accuracy. 1 Introduction Ensemble [26, 42, 51, 67, 70, 70, 71, 73, 90] (aka. model averaging) is one of the oldest and most powerful t”
Illuminating the “black box”: a randomization approach for understanding variable contributions in artificial neural networks (2002, background): “The first approach involves combining different local minima rather than choosing between them, for example, by averaging the outputs of networks using the connection weights corresponding to different local minima (e.g. Wolpert, 1992; Perrone and Cooper, 1993; Ripley, 1995).”
Ensemble Pruning Via Semi-definite Programming (2006, background): “Some previous work has demonstrated that making errors in an uncorrelated manner leads to a low error rate (Hansen and Salamon, 1990; Perrone and Cooper, 1993).”
Scene Classification via a Gradient Boosting Random Convolutional Network Framework (2016, methodology): “The common approaches to an ensemble are plurality voting or majority voting [12] for classification tasks and simple averaging [21] or weighted averaging [22] for regression tasks.”
Fast decorrelated neural network ensembles with random weights (2014, unknown): “E-mail addresses: mal-hamdoosh@students.latrobe.edu.au (M. Alhamdoosh), dh.wang@latrobe.edu.au (D. Wang).”
Improving model accuracy using optimal linear combinations of trained neural networks (1995, methodology): “The General Ensemble Method [6], developed independently, is similar to our constrained MSE-OLC.”
Ensembles of Learning Machines (2002, methodology): “The simplest approach is a greedy one [104], where a new learner is added to the ensemble only if the resulting squared error is reduced, but in principle any optimization technique can be used to select the ”best” component of the ensemble, including genetic algorithms [97].”
Classifier ensembles: Select real-world applications (2008, methodology): “1 This result has an equivalent for regression estimators where a reduction in Mean Square Error by a factor of M can be obtained by pooling M independent estimators [83].”
Soft computing system for bank performance prediction (2008, background): “, simple average [9], weighted average [39] and stacked regression [10]) and non-linear ensemble (e.”
Ensemble of data-driven prognostic algorithms for robust prediction of remaining useful life (2011, methodology): “The ensemble method founds its applications in a wide variety of research fields, such as the development of committees of neural networks [28,29], the metamodeling for the design of modern engineered systems [30–32], the discovery of regulatory motifs in bioinformatics [33], the detection of traffi”