Bayesian Nonparametrics

By Han Lin Shang. 2011

This book consists of a collection of articles concerning Bayesian nonparametric methods. The recent theoretical and practical aspects of Dirichlet processes are discussed. The book covers the theoretical aspects in the first four chapters, while the last four chapters document a wide range of applications from biostatistics to signal processing. The theory of Bayesian nonparametrics presents flexible models, whose complexity increases with increasing amount of data and dimensionality. The theorems of ‘stick-breaking’ representation, Chinese restaurant process, Indian buffet process, and Bernstein–von Mises have been covered repeatedly throughout the first four chapters. On the practical aspects, the Bayesian nonparametric techniques (such as the hierarchical Dirichlet process) have been widely applied to the areas of information retrieval, speaker diarisation, word segmentation, bioinformatics, and microarrays. The contribution of this book is to collect most recent research of Bayesian nonparametric techniques together, with main emphasis on the use of Dirichlet process. The popularity of Dirichlet process is because that the Dirichlet prior is nonparametric and conjugate, thus presenting many opportunities to flexibly model complex data structure. The book incorporates the Bayesian philiosophy into the nonparametric concept. It also introduces Bayesian computational tools (such as the Metropolis-Hasting and Gibbs algorithms) to deal with possibly infinite number of parameters. A statistical software package in R called DRpackage has been introduced to implement Dirichlet process mixture density estimation, Pólya tree priors for density estimation, and nonparametric random effects models including generalized linear models. The authors have fulfilled their main aim which is to introduce Bayesian nonparametrics. However, I feel this book could go even further by combining the Bayesian computational techniques with frequentist nonparametric methods. In other words, could the Bayesian techniques be well suited for solving frequentist nonparametric problems? If so, this would bring the Bayesian philosophy and frequentist philiosophy one step closer. Overall, I enjoyed reading and reviewing this book and I feel that this book gives nice theoretical and practical viewpoints on Bayesian nonparametrics. The use of Bayesian nonparametric techniques is suitable for any statistician, who aims to analyse complex data structures with possibly infinite number of parameters. Although this book concentrates mainly on the Dirichlet process, it serves perfectly as a reference for scientists and graduate students working in the field of Bayesian data analysis.

Subfields: Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Probabilistic & Bayesian Methods

Citations in the corpus

Stochastic variational inference (2012, methodology): “Bayesian nonpara-metrics gives us a collection of ﬂexible models—mixture models, mixed-membership models, factor models, and models with more complex structure—which grow and expand with data (Hjort et al., 2010).”
Relational inductive biases, deep learning, and graph networks (2018, background): “The question of how to build artificial systems which exhibit combinatorial generalization has been at the heart of AI since its origins, and was central to many structured approaches, including logic, grammars, classic planning, graphical models, causal reasoning, Bayesian nonparametrics, and proba”
Philosophy and the practice of Bayesian statistics (2010, unknown): “19 Taken to its limit, then, the idea of continuous model expansion counsels social scientists to pretty much give up using parametric statistical models in favor of nonparametric, infinite-dimensional models, advice which which the ongoing rapid development of Bayesian nonparametrics (Ghosh and Ram”
A note on the multiplicative gamma process (2016, methodology): “Examples include tensor factorization (Dunson and Xing, 2009), matrix decomposition (Mazumder et al., 2010), mixture modeling (Hjort et al., 2010) and factor analysis (Knowles and Ghahramani, 2011).”
POSTERIOR CONSISTENCY IN CONDITIONAL DENSITY ESTIMATION BY COVARIATE DEPENDENT MIXTURES (2011, methodology): “A widely accepted minimal requirement for large-sample behavior of Bayesian nonparametric models is posterior consistency (see Ghosh and Ramamoorthi, 2003, for a textbook treatment).”
Spectral gaps for a Metropolis–Hastings algorithm in infinite dimensions (2011, methodology): “Creating samples from a high dimensional probability distribution is an essential tool in Bayesian inverse problems [Stuart (2010)], Bayesian statistics [Lee (2004)], Bayesian nonparametrics [Hjort et al. (2010)], and conditioned diffusions [Hairer, Stuart and Voss (2007)].”
Infinite Mixture Prototypes for Few-Shot Learning (2019, background): “Inﬁnite mixture models (Hjort et al., 2010) do not require the number of mixture components to be known and ﬁnite.”
VariGrow: Variational Architecture Growing for Task-Agnostic Continual Learning based on Bayesian Novelty (2022, background): “Hence it is important that our model scales accordingly (Hjort et al., 2010).”
MCMC Methods for Functions: ModifyingOld Algorithms to Make Them Faster (2012, background): “INTRODUCTION The use of Gaussian process (or field) priors is widespread in statistical applications (geostatistics [58], non-parametric regression [30], Bayesian emulator modeling [43], density estimation [2] and inverse quantum theory [34] to name but a few substantial areas where they are commonp”
Conditional (2019, unknown): “To further supplement the texture and background information from the source face to the rendered face, we introduce a cross-attention mechanism to accurately match the correspondences between the source face and the rendered face.”
Build, Compute, Critique, Repeat: Data Analysis with Latent Variable Models (2014, methodology): “In fact, we omitted some important components, like Bayesian nonparametric models (Ferguson, 1973; Antoniak, 1974; Teh and Jordan, 2008; Hjort et al., 2010; Gershman and Blei, 2012) that let the data determine the structure of the latent variables, random effects models (Gelman et al.”
Probabilistic machine learning and artificial intelligence (2015, background): “A full discussion of Bayesian non-parametrics is outside the scope of this Review (see refs 9, 31, 32 for this), but it is worth mentioning a few of the key models.”