Gaussian Process Networks

By Nir Friedman, Iftach Nachman. 2000

91 citations recorded by the source corpus.

In this paper we address the problem of learning the structure of a Bayesian network in domains with continuous variables. This task requires a procedure for comparing different candidate structures. In the Bayesian framework, this is done by evaluating the marginal likelihood of the data given a candidate structure. This term can be computed in closed-form for standard parametric families (e.g., Gaussians), and can be approximated, at some computational cost, for some semi-parametric families (e.g., mixtures of Gaussians). We present a new family of continuous variable probabilistic networks that are based on Gaussian Process priors. These priors are semiparametric in nature and can learn almost arbitrary noisy functional relations. Using these priors, we can directly compute marginal likelihoods for structure learning. The resulting method can discover a wide range of functional dependencies in multivariate data. We develop the Bayesian score of Gaussian Process Networks and describe how to learn them from data. We present empirical results on artificial data as well as on real-life domains with non-linear dependencies.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Algorithmic recourse under imperfect causal knowledge: a probabilistic approach (2020, background): “While GPs have previously been studied in a causal context for structure learning [13, 56], estimating treatment effects [2, 43], or learning SCMs with latent variables and measurement error [47], our goal here is to account for the uncertainty over fr in the computation of the posterior over Ur, an”
Distinguishing Cause from Effect Using Observational Data: Methods and Benchmarks (2014, methodology): “This Gaussian score is also strongly related to an empirical-Bayes score originally proposed by Friedman and Nachman (2000).”
On the Identifiability of the Post-Nonlinear Causal Model (2009, positive): “This result is intuitively appealing, and confirms the finding in Friedman and Nachman (2000).”
Testing whether linear equations are causal: A free probability theory approach (2011, background): “In particular one can infer – with some reliability – whether X causes Y or Y causes X, given that either statement is true (e.g. Friedman and Nachman (2000); Kano and Shimizu (2003); Sun et al. (2006); Hoyer et al. (2009); Daniušis et al. (2010); Mooij et al. (2009)).”
Gaussian Process Regression Networks (2011, methodology): “…observation model, by letting z in (1) be t distributed, with minimal changes to the inference procedures; 3) we can transform the components of the product W ( x i ) ˆ f ( x i ) so that the priors on the components of y ( x i ) become copula processes (Wilson and Ghahramani, 2010a) and have…”
Inference of Cause and Effect with Unsupervised Inverse Regression (2015, background): “Non-invertible functional relationships between the observed variables can provide clues to the generating causal model [Friedman and Nachman, 2000].”
Information-geometric approach to inferring causal directions (2012, methodology): “First, the absence of noise makes additive noise model based inference impossible [9], and second, methods that use non-invertibility of the functional relation fail [16].”
Bayesian Gaussian process models : PAC-Bayesian generalisation error bounds and sparse approximations (2003, background): “Friedman and Nachman [60] propose Gaussian process networks to model sets of random variables with complicated nonlinear interactions (and only vague prior knowledge about their characteristics) and show how to learn the network structure.”
Structural Agnostic Modeling: Adversarial Learning of Causal Graphs (2018, methodology): “The extension of the bivariate approaches to the multivariate setting has been tackled by Friedman and Nachman (2000); Bühlmann et al. (2014) assuming additive noise, and identifiability results have been obtained for the causal additive models (CAM) (Bühlmann et al., 2014).”
Distinguishing Cause from Effect Using Observational Data: Methods and Benchmarks (2014, methodology): “This Gaussian score is also strongly related to an empirical-Bayes score originally proposed by Friedman and Nachman (2000).”
Structural Modelling with Sparse Kernels (2002, background): “The Gaussian process network (Friedman and Nachman, 2000) has recently been introduced as a new family of continuous variable probabilistic networks that are based on Gaussian process priors to overcome the limitations of the graphical model.”
Nonlinear causal discovery with additive noise models (2008, background): “As Friedman and Nachman have pointed out [10], non-invertible functional relationships between the observed variables can provide clues to the generating causal model.”
Mixtures of Gaussian Processes (2000, background): “Readers might also be interested in the related and independent paper by Friedman and Nachman (2000) in which those authors used GPR systems (not in form of the MGP) to perform structural learning in Bayesian networks of continuous variables.”
RNADE: The real-valued neural autoregressive density-estimator (2013, methodology): “In more general work, Gaussian processes have been used to model the conditional distributions of a fully visible Bayesian network [12].”
Gaussian Process Structural Equation Models with Latent Variables (2010, methodology): “Following Harrison and Rubinfield, we log-transformed some of the variables: INDUS, DIS, RAD andTAX. ing dataset poses stability problems to density estimation due to discontinuities in the variableRAD, one of the indicators of accessibility (Friedman and Nachman, 2000).”
Distinguishing between cause and effect (2008, methodology): “869×1003 1← 2 1→ 2 Table 2: Baseline results for distinguishing the cause from the effect, using the method of Friedman and Nachman (2000); S denotes the logarithm of the marginal likelihood of the Gaussian Process fit.”
Optimal experimental design via Bayesian optimization: active causal structure learning for Gaussian process networks (2019, background): “To the best of our knowledge, ours is the first work to address the Bayesian experimental design problem for continuous variables, allowing for flexible non-linear relationships through the use of GP networks [31].”