Neural network with ensembles

By E. Kussul, O. Makeyev, T. Baidyk, D. Reyes. 2010

1,765 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Control and Systems Engineering, Information Systems, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Control and Systems Engineering

In Information Systems

In Model Architectures

Citations in the corpus

The Power of Scale for Parameter-Efficient Prompt Tuning (2021, background): “to improve task performance (Hansen and Salamon, 1990), and are useful for estimating model uncertainty (Lakshminarayanan et al.”
Born Again Neural Networks (2018, background): “In a well-known paper on algorithmic modeling [1], Leo Breiman noted that different stochastic algorithmic procedures [2, 3, 4] can lead to diverse models with similar validation performances.”
BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learning (2020, background): “Moreover, ensembles achieve the best performance when each of their members makes independent errors (Goodfellow et al., 2015; Hansen & Salamon, 1990).”
Essentially No Barriers in Neural Network Energy Landscape (2018, methodology): “This type of resilience is exploited and encouraged by procedures such as Dropout (Srivastava et al., 2014) or ensem-bling (Hansen & Salamon, 1990).”
Snapshot Ensembles: Train 1, get M for free (2017, methodology): “Neural network ensembles have been widely studied and applied in machine learning (Hansen & Salamon, 1990; Krogh et al., 1995).”
Data Distillation: Towards Omni-Supervised Learning (2017, methodology): “Ensembling [14] multiple models has been a successful method for improving accuracy.”
Sensitivity of Support Vector Machines to Random Feature Selection in Classification of Hyperspectral Data (2010, neutral): “Studies trying to employ computationally more complex classiﬁers such as neural networks [26] and SVMs [27] in MCSs exist, but they are very limited, particularly in the remote sensing context.”
Simplifying Particle Swarm Optimization (2010, methodology): “It is customary to split the dataset T and use one part during optimization (or training) of the ANN, and the other part to test the generalization ability of the ANN, as a simple form of statistical cross-validation [27].”
Features in Concert: Discriminative Feature Selection meets Unsupervised Clustering (2014, background): “Decades of research in machine learning show that an ensemble can be significantly stronger than an individual classifier in isolation [9, 17], especially when the individual classifiers are diverse and make mistakes on different regions of the input space.”
Scene Classification via a Gradient Boosting Random Convolutional Network Framework (2016, methodology): “The common approaches to an ensemble are plurality voting or majority voting [12] for classification tasks and simple averaging [21] or weighted averaging [22] for regression tasks.”
A Review of Generalized Zero-Shot Learning Methods (2020, neutral): “ensemble models [205] and metalearning strategy [110].”
Multiobjective Intelligent Energy Management for a Microgrid (2013, background): “On the other hand, there is no systematic investigation for those issues; mostly, researchers have adopted trial and error methodology to deal with these inconsistencies [17].”
Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning (2020, background): “Ensemble (Dietterich, 2000; Hansen & Salamon, 1990; Polikar, 2006) is one of the most powerful techniques in practice to improve the performance of deep learning.”
Why M Heads are Better than One: Training a Diverse Ensemble of Deep Networks (2015, methodology): “Krogh et al. [22] and Hansen and Salamon [16] provided theoretical and empirical evidence that diversity in error distributions across member models can boost ensemble performance.”
Automated trading with performance weighted random forests and seasonality (2014, methodology): “Over the last few years, the ensemble approach has been applied with great success in expert learning systems such as decision trees (Breiman, 1996) and neural networks (Hansen and Salamon, 1990).”
Neural network ensemble operators for time series forecasting (2014, methodology): “…of selecting a single NN that may be susceptible to poor initial values (or model setup), it is preferable to consider a combination of diﬀerent NN models (Hansen and Salamon, 1990; Zhang and Berardi, 2001; Versace et al., 2004; Barrow et al., 2010; Crone et al., 2011; Ben Taieb et al., 2012).”
Review on Studies and Advances of Machine Learning Approaches (2013, methodology): “Ensembles of other basic classifiers, such as ensemble of Naive Bayesian, ensemble of Decision Tree [26], ensemble of NNs [27-29], and ensemble of KNN [30].”
Neural Networks and Deep Learning (2018, neutral): “Ensemble methods for neural networks are proposed in [170].”