Generalized Product of Experts for Automatic and Principled Fusion of Gaussian Process Predictions

By Yanshuai Cao, David J. Fleet. 2014

204 citations recorded by the source corpus.

In this work, we propose a generalized product of experts (gPoE) framework for combining the predictions of multiple probabilistic models. We identify four desirable properties that are important for scalability, expressiveness and robustness, when learning and inferring with a combination of multiple models. Through analysis and experiments, we show that gPoE of Gaussian processes (GP) have these qualities, while no other existing combination schemes satisfy all of them at the same time. The resulting GP-gPoE is highly scalable as individual GP experts can be independently learned in parallel; very expressive as the way experts are combined depends on the input rather than fixed; the combined prediction is still a valid probabilistic model with natural interpretation; and finally robust to unreliable predictions from individual experts.

Subfields: Artificial Intelligence, Statistical and Nonlinear Physics, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistical and Nonlinear Physics

In Signal Processing

Citations in the corpus

When Gaussian Process Meets Big Data: A Review of Scalable GPs (2018, background): “To alleviate the overconfident uncertainty, the generalized PoE (GPoE) [143] introduces a varying weight β∗i , which is defined as the difference in the differential entropy between the expert’s prior and posterior, to increase or decrease the importance of experts based on their prediction uncertai”
Multimodal Generative Models for Scalable Weakly-Supervised Learning (2018, unknown): “In future work, we would like to explore a product of mixture of Gaussian posteriors, and apply our method to more real-world tasks, such as image captioning and tactile-visual learning.”
Distributed Gaussian Processes (2015, methodology): “…the following, we assume that a set of M GP experts has been trained according to Section 3.1 and detail how the PoE (Ng & Deisenroth, 2014), the generalised PoE (Cao & Fleet, 2014) and the Bayesian Committee Machine (Tresp, 2000) combine predictions of the GP experts to form an overall prediction.”