Bias/Variance Analyses of Mixtures-of-Experts Architectures

By Robert A. Jacobs. 1997

Subfields: Model Architectures, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Model Architectures

In Statistical Learning Theory

Citations in the corpus

DSelect-k: Differentiable Selection in the Mixture of Experts with Applications to Multi-Task Learning (2021, neutral): “, see Jordan and Jacobs [17], Jacobs [13], Jiang”
ExpertMatcher: Automating ML Model Selection for Clients using Hidden Representations (2019, background): “The Expert matching problem as described by Sharma et al. (2019) is to assign input data from a client to the most appropriate expert model without any information other than the input data itself.”
Layered Ensemble Architecture for Time Series Forecasting (2016, methodology): “tion [35], [61], double fault [62] and disagreement [62].”
Neural networks for classification: a survey (2000, background): “Jacobs [85] studies various properties of bias and variance components for mixtures-of-experts architectures.”
Twenty Years of Mixture of Experts (2012, background): “Jacobs [79] analyzed the bias and variance of ME architectures and showed that ME produces biased experts whose estimates are negatively correlated.”
Mixture of experts: a literature survey (2012, unknown): “It was shown that in contrast to common combining methods that produce unbiased experts whose estimation errors are uncorrelated, the ME method produce biased experts whose estimates are negatively correlated (Jacobs 1997).”
Evolutionary ensembles with negative correlation learning (2000, methodology): “Since the early 1990's, algorithms based on similar ideas have been developed in many di erent but related forms, such as NN ensembles [5, 2], mixtures of experts [6, 7, 8, 9], and various boosting and bagging methods [10, 11, 12].”