Bayesian Methods for Mixtures of Experts

By Steve R. Waterhouse, David John Cameron MacKay, Anthony J. Robinson. 1995

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Variational Inference: A Review for Statisticians (2016, background): “…published in 1993) made important connections to the expectation maximization (EM) algorithm (Dempster, Laird, and Rubin 1977), which then led to a variety of variational inference algorithms for other types of models (Waterhouse, MacKay, and Robinson 1996; MacKay 1997; Barber and Bishop 1998).”
The Variational Gaussian Process (2015, background): “Originally developed in the 1990s (Hinton & Van Camp, 1993; Waterhouse et al., 1996; Jordan et al., 1999), variational inference has enjoyed renewed interest around developing scalable optimization for large datasets (Hoffman et al., 2013), deriving generic strategies for easily ﬁtting many models…”
Edward: A library for probabilistic modeling, inference, and criticism (2016, background): “As a first consequence, we focus on variational inference (Hinton and van Camp, 1993; Waterhouse et al., 1996; Jordan et al., 1999).”
Variational approximations in Bayesian model selection for finite mixture distributions (2007, methodology): “Waterhouse et al. (1996) proposed estimating the parameters and hyperparameters of a mixture model by using a Bayesian framework based on the variational approximation as an alternative to the maximum likelihood approach which tends to over-fit the model. Attias (1999) extended the variational Bayes”
A family of algorithms for approximate Bayesian inference (2001, methodology): “This approach was used explicitly by Jaakkola & Jordan (1999a; 1999b) and implicitly by Waterhouse et al. (1995); Attias (1999); Ghahramani & Beal (1999), under the name \variational Bayes."”
Importance Weighted Hierarchical Variational Inference (2019, background): “Bayesian Inference is an important statistical tool.”
Mixture of experts: a literature survey (2012, unknown): “Different types of learners and training algorithms were also employed for the experts and gating network learning in the MILE (Waterhouse et al. 1996; Hansen 1999; Ebrahimpour et al. 2008d).”
Bayesian Treed Models (2002, background): “Waterhouse, MacKay, and Robinson (1996) consider a tree-structured mixture of experts model, in which the individual experts are linear models.”
Twenty Years of Mixture of Experts (2012, methodology): “1) Inference. a) EM-based methods: IRLS [2], generalized EM [50], Newton–Raphson [51], ECM [52], singleloop EM [53]. b) Variational [7], [43], [45], [54]. c) Sampling [6], [18], genetic training [25].”
Variational inference for visual tracking (2003, background): “In this case it can be shown (see [13]) that the approximate posterior distributions that maximise the lower bound are of the form”
When Gaussian Process Meets Big Data: A Review of Scalable GPs (2018, methodology): “s extended to a tree-structured hierarchical architecture [135]; (ii) the Bayesian approach is employed instead of the simple maximum likelihood to get rid of over-ﬁtting and noiselevel underestimate [137], [138]; (iii) the t-distribution is considered to handle the outliers [139]; and ﬁnally (iv) i”
An Introduction to Hidden Markov Models and Bayesian Networks (2001, unknown): “k=1 (h (m) 1,k π (m) k ) S (m) 1,k (58)”
A Bayesian network framework for relational shape matching (2003, methodology): “While they maximized model likelihood to learn new categories, we use variational Bayesian methods by incorporating “general” knowledge of object categories [11, 12, 14].”
Propagation Algorithms for Variational Bayesian Learning (2000, background): “It has since been extended to models with hidden variables and the restrictions on Qo(O) and Qx(x) have been removed in certain models to allow arbitrary distributions [11] [8] [3] [1] [5].”