Optimal Linear Combination of Neural Networks for Improving Classification Performance

By Naonori Ueda. 2000

254 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Control and Systems Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

Citations in the corpus

Deep learning in neural networks: An overview (2014, methodology): “, 2011b) are committees (Breiman, 1996; Schapire, 1990; Wolpert, 1992; Hashem and Schmeiser, 1992; Ueda, 2000; Dietterich, 2000a) of GPU-MPCNNs with simple democratic output averaging.”
Multi-column deep neural network for traffic sign classification (2012, methodology): “For handwritten digits it was shown (Meier et al., 2011), that simply averaging the outputs of many DNN generalizes better on the test set than a combination of all the DNN with weights over a set ( Hashem & Schmeiser, 1995 ; Ueda, 2000).”
Time series forecasting with neural network ensembles: an application for exchange rate prediction (2001, background): “15,18,19 with no consensus on the best way to assign weights.”
Neural networks for classification: a survey (2000, background): “But combining can also be done with weighted averaging that treats the contribution or accuracy of component classifiers differently [68], [67], [84].”
Multiscale Convolutional Neural Networks for Vision-Based Classification of Cells (2012, background): “For handwritten digits, it was shown [9] that a simple average gives better classification results than a linear combination whose weights learned over a cross–validation set [10].”
Extracting symbolic rules from trained neural network ensembles (2003, methodology): “Ueda [43] exploits optimal linear weights to combine individual predictions based on statistical pattern recognition theory.”
A theoretical and experimental analysis of linear combiners for multiple classifier systems (2005, methodology): “One regularization method suggested for avoiding this problem is to constrain the weights to be nonnegative [17], [11], [4], [13].”
Ensembling neural networks: Many could be better than all (2002, methodology): “Ueda [43] exploits optimal linear weights to combine component predictions based on statistical pattern recognition theory.”
Convolutional Neural Network Committees for Handwritten Character Classification (2011, background): “Furthermore, we simply average individual committee member outputs, instead of optimizing their combinations [15], [16], which would cost additional valuable training data.”