Neural Network Ensembles, Cross Validation, and Active Learning

By Anders Krogh, Jesper Vedelsby. 1994

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

FOSTER: Feature Boosting and Compression for Class-Incremental Learning (2022, methodology): “Given that the original model can handle old classes well, combining the two will form a strong classifier for both new and old classes [30].”
BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learning (2020, background): “It is shown that ensembles of models perform at least as well as its individual members and diverse ensemble members lead to better performance (Krogh & Vedelsby, 1995).”
Snapshot Ensembles: Train 1, get M for free (2017, methodology): “Neural network ensembles have been widely studied and applied in machine learning (Hansen & Salamon, 1990; Krogh et al., 1995).”
Safe Visual Navigation via Deep Learning and Novelty Detection (2017, background): “Recognizing this limitation, some work has attempted to quantify uncertainty with Bayesian neural networks [20, 5], by training ensembles of networks on re-sampled versions of the training dataset [26, 17, 8, 18], or by using dropout to train an implicit family of models [9].”
Boosting Neural Networks (2000, background): “It is well known that it is often possible to increase the accuracy of a classifier by averaging the decisions of an ensemble of classifiers (Perrone, 1993; Krogh & Vedelsby, 1995).”
Exploiting the past and the future in protein secondary structure prediction (1999, methodology): “In a second set of experiments based on the 824 sequences , we combined several BRNNs, trained with 1/3– 2/3 data splitting, to form an ensemble, as in Krogh and Vedelsby (1995), using a simple averaging scheme.”
A hybrid ARIMA and support vector machines model in stock price forecasting (2005, background): “Theoretically as well as empirically, hybridizing two dissimilar models reduces forecasting errors [31,32].”
On Combining Classifiers (1998, methodology): “The combination of ensembles of neural networks (based on different initialisations), has been studied in the neural network literature, e.g., [11], [4], [5], [10], [15], [18].”
Credit risk assessment with a multistage neural network ensemble learning approach (2008, methodology): “One important ingredient needed to accomplish this goal is to seek an accurate classifier in order to categorize new applicants or existing customers as good or bad.”
Layered Ensemble Architecture for Time Series Forecasting (2016, methodology): “Variance is usually used in regression problems to select the diverse networks [48] and”
BERT Loses Patience: Fast and Robust Inference with Early Exit (2020, background): “Also, since PABEE comprehensively considers results from multiple classifiers, it can also benefit from an ensemble learning [34] effect.”
Ensemble-based classifiers (2010, background): “Diversity is a crucial condition for obtaining accurate ensembles (Tumer and Ghosh 1996; Krogh and Vedelsby 1995; Kuncheva 2005; Kuncheva and Whitaker 2003).”
Cluster ensembles --- a knowledge reuse framework for combining multiple partitions (2003, background): “Also, Mehrotra (1999) has proposed a multi-viewpoint clustering, where several clusterings are used to semi-automatically structure rules in a knowledge base.”
Active Hidden Markov Models for Information Extraction (2001, background): “Such algorithms are known for classification (e.g., [3]), clustering [8], and regression [ 10 ]; here, we present the first algorithm for active learning of hidden Markov models.”
Data Distillation for Controlling Specificity in Dialogue Generation (2017, methodology): “…is also related to data manipulation strategies such as boosting (Breiman, 1996b), a type of ensemble method (Dietterich, 2002; Zhou et al., 2002; Krogh et al., 1995) that uses subsets of the original data to produce a series of models and then ”boosts” their performance by combining them…”