Bayesian Treed Gaussian Process Models With an Application to Computer Modeling

By Robert B. Gramacy, Herbert K. H. Lee. 2007

734 citations recorded by the source corpus.

Motivated by a computer experiment for the design of a rocket booster, this article explores nonstationary modeling methodologies that couple stationary Gaussian processes with treed partitioning. Partitioning is a simple but effective method for dealing with nonstationarity. The methodological developments and statistical computing details that make this approach efficient are described in detail. In addition to providing an analysis of the rocket booster simulator, we show that our approach is effective in other arenas as well.

Subfields: Artificial Intelligence, Environmental Engineering, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/2c1ac493143df6a38197a2d2cef58e45b75df9ca

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Environmental Engineering

In Statistics and Probability

Citations in the corpus

Hierarchical Nearest-Neighbor Gaussian Process Models for Large Geostatistical Datasets (2014, methodology): “Another choice for S could be based upon configurations for treedGaussian processes (Gramacy and Lee 2008).”
Review of surrogate modeling in water resources (2012, methodology): “…1991; Jin et al., 2001], high-dimensional model representation [Rabitz et al., 1999; Ratto et al., 2007; Sobol, 2003], treed Gaussian processes [Gramacy and Lee, 2008], Gaussian emulator machines [Kennedy and O’Hagan, 2001; O’Hagan, 2006], smoothing splines ANOVA models [Gu, 2002; Ratto and…”
Diagnostics for Gaussian Process Emulators (2009, methodology): “For instance, for the nonstationary model, the treed Gaussian process could be used (Gramancy and Lee 2008).”
Optimization Under Unknown Constraints (2010, background): “We follow Gramacy and Lee (2008) in specifying that K(·, ·) have the form K(x, x′|g) = K∗(x, x′) + ηδx,x′, where δ·,· is the Kronecker delta function, and K∗ is a true correlation function.”
Bayesian Regression Tree Models for Causal Inference: Regularization, Confounding, and Heterogeneous Effects (with Discussion) (2020, positive): “In terms of implementation, this paper builds explicitly on the work of Chipman et al. (2010); see also Gramacy and Lee (2008) and Murray (2017).”
A Bayesian nonparametric approach to marginal structural models for point treatments and a continuous or survival outcome. (2017, methodology): “Another option is to use an alternative to the GP model, such as treed GP models (Gramacy and Lee, 2008).”
Predicting Vehicle Crashworthiness: Validation of Computer Models for Functional and Hierarchical Data (2009, methodology): “…use of Gaussian process response-surface approximations to a computer model, following on work in Sacks et al. (1989), Currin et al. (1991), Welch et al. (1992), and Morris, Mitchell, and Ylvisaker (1993 Campbell (2006), Gramacy and Lee (2006), and Gramacy and Lee (2008) are more recent references.”
Globally Approximate Gaussian Processes for Big Data With Application to Data-Driven Metamaterials Design (2019, background): “One avenue of research has explored partitioning the input space (and hence the training data) via, e.g., trees [19] or Voronoi cells [20], and ﬁ tting an independent GP to each partition.”
A Scalable Hierarchical Gaussian Process Classifier (2019, methodology): “Many MoGPEs have been proposed for regression [25]–[28], [30]– [32], [60], in which inference involves simultaneous learning of the GP experts and the gating network.”
Surrogates: Gaussian Process Modeling, Design, and Optimization for the Applied Sciences (2020, positive): “…ideas, like the treed gaussian process that serves a good example of how a “perverse” marriage was made: “Bayesian treed gaussian process (TGP) models (Gramacy and Lee 2008) can offer the best of both worlds, marrying the smooth global perspective of an infinite basis expansion, via GPs, with the…”
Bayesian regression tree models for causal inference: regularization, confounding, and heterogeneous effects (2017, positive): “BART has been used successfully in applications, for example Green and Kern (2012), Hill et al. (2013), Kern et al. (2016), and Sivaganesan, Müller and Huang (2017).”
Bayesian and Empirical Bayesian Forests (2015, methodology): “The BCART authors propose linear regression leaves, while the Treed Gaussian Processes (TGP) of Gramacy & Lee (2008) use Gaussian process regression at each leaf.”
When Gaussian Process Meets Big Data: A Review of Scalable GPs (2018, background): “Efficiently, local approximations only need to tackle a local expert with m0 (m0 n) data points at each time [19], [20].”
Survey of multifidelity methods in uncertainty propagation, inference, and optimization (2018, methodology): “These local models are obtained via the treed Gaussian-process implementation [83] that partitions the input space into several regions and builds separate Gaussian-process models in each region.”
Modeling an augmented Lagrangian for improved blackbox constrained optimization (2014, methodology): “Modern GP methods accommodate nonstationarity (Schmidt and O’Hagan, 2003; Pa-ciorek and Schervish, 2006) and even regime-changing behavior (Gramacy and Lee, 2008).”
Implementation and evaluation of nonparametric regression procedures for sensitivity analysis of computationally demanding models (2009, background): “Here we focus on one very popular class of covariances, the powered exponential family ([45], [11]).”
Learning about physical parameters: the importance of model discrepancy (2014, methodology): “Since then, their inferential ideas and modeling framework have been widely adopted and further developed, e.g. Higdon et al (2004, 2008), Qian and Wu (2008), Goldstein and Rougier (2009), Bayarri et al (2007), Gramacy and Lee (2008).”
Cases for the nugget in modeling computer experiments (2010, methodology): “For example, the resulting multivariate normal likelihood can be maximized numerically, although the presence of the nugget may lead to a bimodal surface (Gramacy and Lee, 2008b).”
Dynamic Trees for Learning and Design (2009, methodology): “(17) Expressions for ∆σ(2) x(x ′ | Tt) may be obtained as a special case of the results given by Gramacy and Lee (2009). Clearly, ∆σ(2) x(x ′ | Tt) = 0 if x and x′ are allocated to different leaves of Tt.”