Dirichlet Process Mixtures of Generalized Linear Models

By Lauren Hannah, David M. Blei, Warrren B Powell. 2009

202 citations recorded by the source corpus.

We propose Dirichlet Process mixtures of Generalized Linear Models (DP-GLM), a new class of methods for nonparametric regression. Given a data set of input-response pairs, the DP-GLM produces a global model of the joint distribution through a mixture of local generalized linear models. DP-GLMs allow both continuous and categorical inputs, and can model the same class of responses that can be modeled with a generalized linear model. We study the properties of the DP-GLM, and show why it provides better predictions and density estimates than existing Dirichlet process mixture regression models. We give conditions for weak consistency of the joint distribution and pointwise consistency of the regression estimate.

Subfields: Artificial Intelligence, Statistics and Probability, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

In Probabilistic & Bayesian Methods

Citations in the corpus

A Tutorial on Bayesian Nonparametric Models (2011, methodology): “Recently, another nonparametric approach to supervised learning has been developed, based on the CRP mixture model (Shahbaba and Neal, 2009; Hannah et al., 2010).”
Randomized Optimum Models for Structured Prediction (2012, background): “Many regression models could be used for RandOMs, e.g., Hannah et al. (2011); however, for concreteness, the reader may think of the model in terms of simple basis function regression with spherical Gaussian noise and the mean constrained to the unit hypercube by the logistic function: where σ ( u…”
Bayesian Nonparametric Modeling for Multivariate Ordinal Regression (2014, methodology): “Other related nonparametric approaches to regression which model the covariates as random and deal with discrete responses include Shahbaba and Neal (2009), Dunson and Bhat-tacharya (2010), Hannah et al. (2011), and Papageorgiou et al. (2014).”
Joint Learning of Set Cardinality and State Distribution (2017, methodology): “Approaches such as mixture models (Blei, Ng, and Jordan 2003; Hannah, Blei, and Powell 2011; Tran et al. 2016), learning distributions from a set of samples (Muandet et al. 2012; Oliva, Póczos, and Schneider 2013), model-based multiple instance learning (Vo et al. 2017) and novelty detection from…”
Learn to Predict Sets Using Feed-Forward Neural Networks (2020, background): “Approaches such as mixture models [31], [39], [40], learning distributions from a set of samples [41], [42], model-based multiple instance learning [43], and novelty detection from point pattern data [44], can be counted as few out many examples that use point patterns or sets as input or output and”
Martingale posterior distributions (2021, methodology): “(1996); Shahbaba and Neal (2009); Hannah et al. (2011), where the prior on f0(yi,xi) is a DPMM. The second type of assumption, which we call the ‘conditional method’, is the more common framework. Here we assume that x1:n are fixed design points and the randomness arises from the response y1:n, so w”