Multiple Classifier Systems

By Thomas G. Dietterich. 2000

6,009 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Evaluation & Benchmarks

Citations in the corpus

Distilling the Knowledge in a Neural Network (2015, background): “A very simple way to improve the performance of almost any machine learning algorithm is to train many different models on the same data and then to average their predictions [3].”
Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time (2022, methodology): “For the ensemble baselines (Dietterich, 2000; Lakshmi-narayanan et al., 2017) we ensemble the logits (unormalized outputs) of models as in Gontijo-Lopes et al. (2022).”
Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles (2016, background): “It has long been observed that ensembles of models improve overall performance (see (Dietterich, 2000) for a review).”
Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data (2016, background): “In this strategy, ﬁrst, an ensemble (Dietterich, 2000) of teacher models is trained on disjoint subsets of the sensitive data.”
Machine Learning Methods That Economists Should Know About (2019, methodology): “, Dietterich [2000]). In many cases a single model or algorithm does not perform as well as a combination of possibly quite different models, averaged using weights (sometimes called votes) obtained by optimizing out-of-sample performance. A striking example is the Netflix Prize Competition (Bennett”
Continual Learning with Deep Generative Replay (2017, background): “Note that the term scholar differs from standard notion of teacher-student framework of ensemble models [5], in which the networks either teach or learn only.”
FEED: Feature-level Ensemble for Knowledge Distillation (2019, methodology): “Model ensemble (Dietterich 2000) is one of the popular regularization methods, which has been used as a way of alleviating the problem of overfitting in a single model.”
Bayesian Nonparametric Federated Learning of Neural Networks (2019, methodology): “Ensemble methods (Dietterich, 2000; Breiman, 2001) are a classic approach for combining predictions of multiple learners.”
Deep learning in neural networks: An overview (2014, methodology): “y contest-winningFNNs (Sec. 5.20, Sec. 5.21). Multi-Column (MC)GPU-MPCNNs (Ciresan et al., 2011b) are committees (Breiman, 1996; Schapire, 1990; Wolpert, 1992; Hashem and Schmeiser, 1992; Ueda, 2000; Dietterich, 2000a) of GPU-MPCNNs with simple democratic output averaging. Several MPCNNs see the sam”
Opening black box Data Mining models using Sensitivity Analysis (2011, methodology): “To increase interpretability from black box DM models, there are two main approaches: extraction of rules and use of visualization techniques.”
BoostClean: Automated Error Detection and Repair for Machine Learning (2017, methodology): “Although there are many possible algorithms [18], we use a powerful technique called Boosting [20] that composes a set of weak learners into a strong learner.”
Machine Unlearning (2019, methodology): “At inference, the test point is fed to each constituent and all the constituents’ responses are aggregated, similar to the case of ML ensembles [39].”
BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learning (2020, background): “Related work on ensembles : Ensembles have been studied extensively for improving model performance (Hansen & Salamon, 1990; Perrone & Cooper, 1992; Dietterich, 2000; Maclin & Opitz, 1999).”
Demystifying Parallel and Distributed Deep Learning (2018, neutral): “Ensemble learning has been used extensively in machine learning before the deep learning era [66] as a form of boosting, and typically increases the overall accuracy over a single model.”
Token-Level Ensemble Distillation for Grapheme-to-Phoneme Conversion (2019, background): “Model ensemble can incorporate the advantages of individual models, and reduce the effect of overﬁtting in a spirit of the bag-ging method [26].”
A survey and critique of multiagent deep reinforcement learning (2018, background): “vation [196, 339, 340, 341, 342] and hierarchical learning [343, 344, 111] have been proposed”
Automated versus Do-It-Yourself Methods for Causal Inference: Lessons Learned from a Data Analysis Competition (2017, methodology): “Then either the estimate from the best method is chosen or estimates from several methods are combined in a weighted average where the weights are based on relative performance (Dietterich, 2000).11”