Posterior consistency of Gaussian process prior for nonparametric binary regression

By Subhashis Ghosal, Anindya Roy. 2006

203 citations recorded by the source corpus.

Consider binary observations whose response probability is an unknown smooth function of a set of covariates. Suppose that a prior on the response probability function is induced by a Gaussian process mapped to the unit interval through a link function. In this paper we study consistency of the resulting posterior distribution. If the covariance kernel has derivatives up to a desired order and the bandwidth parameter of the kernel is allowed to take arbitrarily small values, we show that the posterior distribution is consistent in the L 1 -distance. As an auxiliary result to our proofs, we show that, under certain conditions, a Gaussian process assigns positive probabilities to the uniform neighborhoods of a continuous function. This result may be of independent interest in the literature for small ball probabilities of Gaussian processes.

Subfields: Artificial Intelligence, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/7b5cca2eafa3d2485475d0564630b1af621c68a1

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Safe Exploration in Finite Markov Decision Processes with Gaussian Processes (2016, methodology): “Based on this confidence within some safe set S, states with small distance to S can be classified to satisfy the safety constraint using the Lipschitz continuity of r(·).”
Multi-fidelity Gaussian Process Bandit Optimisation (2016, background): “It is satisfied for four times differentiable kernels such as the SE kernel and Matérn kernel when ν > 2 (Ghosal and Roy, 2006).”
Generalized Gaussian Process Regression Model for Non-Gaussian Functional Data (2014, background): “This kind of problems for GPR related models has received increasing attention in recent years, see, for example, Choi (2005), Ghosal and Roy (2006), and Seeger, Kakade, and Foster (2008).”
High Dimensional Bayesian Optimisation and Bandits via Additive Models (2015, background): “We deﬁne the Additive Gaussian Process Upper Conﬁdence Bound ( Add-GP-UCB ) to be We immediately see that we can write (cid:101) ϕ t as a sum of functions on orthogonal domains: j ) ) .”
Gaussian Process Bandit Optimisation with Multi-fidelity Evaluations (2016, unknown): “10 0 10 1 10 2 BoreHole-8D, M = 2, Costs = [1; 10]”
Nonparametric binary regression using a Gaussian process prior (2007, methodology): “Under certain conditions, Ghosal and Roy [7] have recently established the posterior consistency of the Gaussian process prior for binary regression.”
Bayesian inference with rescaled Gaussian process priors (2007, methodology): “…model for a regression function (see e.g. [6], [22], [16]); after a monotonic transformation to the unit interval it can be used in the setting of classiﬁcation (e.g. [16], [1], [5]); and after exponentiation and renormalization it becomes a model for density estimation (e.g. [11], [9], [18]).”
Convolutional Gaussian Processes (2017, background): “Because of this, much attention over recent years has been devoted to scaling GP inference to large datasets through sparse approximations [2, 13, 14], minibatch-based optimisation [3], exploiting structure in the covariance matrix [e.”
Remarks on consistency of posterior distributions (2008, methodology): “Instead, those results for independent but non-identically distributed models as in Amewou-Atisso et al. [9], Choudhuri et al. [1], Ghosal and Roy [16] and Choi and Schervish [8], tried to establish the existence of uniformly consistent tests directly, which makes the numerator in the ratio of two i”
Information-Theoretic Regret Bounds for Gaussian Process Optimization in the Bandit Setting (2009, background): “Theorem 5 of Ghosal & Roy (2006) states that if derivatives up to fourth order exists for ( x , x (cid:48) ) (cid:55)→ k ( x , x (cid:48) ), then f is almost surely continuously diﬀerentiable, with ∂f/ ( ∂x j ) distributed as Gaussian processes again.”
Parallelised Bayesian Optimisation via Thompson Sampling (2018, background): “Assumption 8 (Gradients of GP Sample Paths [10]).”