Linearly Combining Density Estimators via Stacking

By Padhraic Smyth, David Hilton Wolpert. 1999

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Evaluation & Benchmarks

Citations in the corpus

Neural Autoregressive Distribution Estimation (2016, background): “A usual approach to improve on a particular estimator is to construct an ensemble of multiple, strong but diﬀerent estimators, e.g. using bagging (Ormoneit and Tresp, 1995) or stacking (Smyth and Wolpert, 1999).”
A Deep and Tractable Density Estimator (2013, background): “A reliable approach to improve on some given estimator is to instead construct an ensemble of multiple, strong but different estimators, e.g. with bagging (Ormoneit & Tresp, 1996) or stacking (Smyth & Wolpert, 1999).”
An Ensemble of Epoch-Wise Empirical Bayes for Few-Shot Learning (2019, methodology): “radient Tree Boosting [62]. Stacking combines multiple models by learning a combiner like a logistic regression model. It applies to both supervised learning tasks [17]–[19] and unsupervised learning [63]. Bootstrap aggregating (bagging) builds an ensemble using models generated in parallel to reduc”
Deep Boosting (2014, methodology): “1 -regularized variants.”
Connectionist multivariate density-estimation and its application to speech synthesis (2016, methodology): “In all our experiments using ensembles, we chose the orderings of variables for each of the components at random, but more elaborate techniques could be used (Smyth and Wolpert, 1999).”
Random forests: from early developments to recent advancements (2014, methodology): “Stacking has been successfully used on both supervised learning tasks (regression) (Breiman, 1996b) and unsupervised learning (density estimation) (Smyth & Wolpert, 1999).”
Fast likelihood-free cosmology with neural density estimators and active learning (2019, methodology): “Stacked density estimators constructed this way are found to outperform a single best density estimator chosen from an ensemble of fits (Smyth & Wolpert 1998, 1999), and stacking ensembles of trained neural networks is common practice in machine learning for improving predictive accuracy.”
Meta Net: A New Meta-Classifier Family (2013, background): “(b) Another strategy is that of mixing many different algorithms from the same training dataset, Stacking Scheme (Wolpert 1992; Smyth and Wolpert 1999).”
FedBoost: Communication-Efficient Algorithms for Federated Learning (2020, unknown): “In the standard supervised learning setting, they include prominent techniques such as bagging, boosting, stacking, error- correction techniques, Bayesian averaging, or other averaging schemes (Breiman, 1996; Freund et al., 1999; Smyth & Wolpert, 1999; MacKay, 1991; Freund et al., 2004).”