Additive Kernels for Gaussian Process Modeling

By Nicolas Durrande, David Ginsbourger, O. Roustant. 2011

28 citations recorded by the source corpus.

Gaussian Process (GP) models are often used as mathematical approximations of computationally expensive experiments. Provided that its kernel is suitably chosen and that enough data is available to obtain a reasonable fit of the simulator, a GP model can beneficially be used for tasks such as prediction, optimization, or Monte-Carlo-based quantification of uncertainty. However, the former conditions become unrealistic when using classical GPs as the dimension of input increases. One popular alternative is then to turn to Generalized Additive Models (GAMs), relying on the assumption that the simulator's response can approximately be decomposed as a sum of univariate functions. If such an approach has been successfully applied in approximation, it is nevertheless not completely compatible with the GP framework and its versatile applications. The ambition of the present work is to give an insight into the use of GPs for additive models by integrating additivity within the kernel, and proposing a parsimonious numerical method for data-driven parameter estimation. The first part of this article deals with the kernels naturally associated to additive processes and the properties of the GP models based on such kernels. The second part is dedicated to a numerical procedure based on relaxation for additive kernel parameter estimation. Finally, the efficiency of the proposed method is illustrated and compared to other approaches on Sobol's g-function.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

In Statistics and Probability

Citations in the corpus

Deep Gaussian Processes (2012, background): “Model selection by our variational bound shows that a five layer hierarchy is justified even when modelling a digit data set containing only 150 examples.”
Stochastic Variational Deep Kernel Learning (2016, methodology): “…by weights w , is applied to the observed input variable x , to produce Q features at the ﬁnal layer L , h ( L ) 1 , . . . , h ( L ) Q ; 2) J Gaussian processes, with base kernels k 1 , . . . , k J , are applied to subsets of these features, corresponding to an additive GP model [e.g., 8].”
Gaussian Process Kernels for Pattern Discovery and Extrapolation (2013, background): “Sophisticated kernels are most often achieved by composing together a few standard kernel functions (Ar-chambeau & Bach, 2011; Durrande et al., 2011; G¨onen & Alpaydın, 2011; Rasmussen & Williams, 2006).”
Gaussian Process Covariance Kernels for Pattern Discovery and Extrapolation (2013, background): “Sophisticated kernels are most often achieved by composing together a few standard kernel functions (Archambeau and Bach, 2011; Durrande et al., 2011; Gönen and Alpaydın, 2011; Rasmussen and Williams, 2006).”
Variational Auto-encoded Deep Gaussian Processes (2015, background): “Indeed, even though sophisticated deﬁnitions and combinations of covariance functions can lead to powerful models (Durrande et al., 2011; G ¨ onen & Alpaydin, 2011; Hensman et al., 2013; Duvenaud et al., 2013; Wilson & Adams, 2013), the assumption about joint normal distribution of instantiations…”