Neural network ensembles: evaluation of aggregation algorithms

By Pablo M. Granitto, Pablo F. Verdes, H. Alejandro Ceccatto. 2005

173 citations recorded by the source corpus.

Ensembles of artificial neural networks show improved generalization capabilities that outperform those of single networks. However, for aggregation to be effective, the individual networks must be as accurate and diverse as possible. An important problem is, then, how to tune the aggregate members in order to have an optimal compromise between these two conflicting conditions. We present here an extensive evaluation of several algorithms for ensemble construction, including new proposals and comparing them with standard methods in the literature. We also discuss a potential problem with sequential aggregation algorithms: the non-frequent but damaging selection through their heuristics of particularly bad ensemble members. We introduce modified algorithms that cope with this problem by allowing individual weighting of aggregate members. Our algorithms and their weighted modifications are favorably tested against other methods in the literature, producing a sensible improvement in performance on most of the standard statistical databases used as benchmarks.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Statistics and Probability

Citations in the corpus

Distributed learning of deep neural network over multiple agents (2018, background): “A reasonable extension to this approach can be to train classiﬁers have also been shown to be more secure against network copy attacks and have also been shown to perform better in real world applications [40].”
A Survey on ensemble learning under the era of deep learning (2021, unknown): “It is generally believed that the base learners used for ensemble should have sufficient diversity and the prediction of a single base learner should be as accurate as possible (Granitto et al. 2005; Zhou 2009, 2012).”
End-to-End Evaluation of Federated Learning and Split Learning for Internet of Things (2020, background): “Ensemble classifiers taking advantage of predictions from multiple models are beneficial for critical applications; for example, they have shown to be more secure against network copy attacks in addition to better prediction performance [22].”
Ensembles for feature selection: A review and future trends (2019, unknown): “…scienti(cid:12)c literature have explored those combination methods [25, 9, 26, 27], and also di(cid:11)erent strategies implying linear and non linear weighting of the base classi(cid:12)ers [28, 29], using genetic algorithms [30], their relation with the base classi(cid:12)ers chosen [31], etc.”