The nested chinese restaurant process and bayesian nonparametric inference of topic hierarchies

By David M. Blei, T. Griffiths, Michael I. Jordan. JACM, 2007

731 citations recorded by the source corpus.

We present the nested Chinese restaurant process (nCRP), a stochastic process that assigns probability distributions to ensembles of infinitely deep, infinitely branching trees. We show how this stochastic process can be used as a prior distribution in a Bayesian nonparametric model of document collections. Specifically, we present an application to information retrieval in which documents are modeled as paths down a random tree, and the preferential attachment dynamics of the nCRP leads to clustering of documents according to sharing of topics at multiple levels of abstraction. Given a corpus of documents, a posterior inference algorithm finds an approximation to a posterior distribution over trees, topics and allocations of words to levels of the tree. We demonstrate this algorithm on collections of scientific abstracts from several journals. This model exemplifies a recent trend in statistical machine learning—the use of Bayesian nonparametric methods to infer distributions on flexible data structures.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/a36a4755cb17f8add5b11cbc58e5901e6a5c1d0a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Nonparametric Variational Auto-Encoders for Hierarchical Representation Learning (2017, methodology): “In particular, we employ nested Chinese Restaurant Process (nCRP) [2], a stochastic process allowing infinitely deep and branching trees for representing the data.”
Graph-Sparse LDA: A Topic Model with Structured Sparsity (2014, methodology): “The use of a human-provided structure to induce interpretability distinguishes GS-LDA from other hierarchical topic models where the structure is learned (Blei, Griffiths, and Jordan 2010; Chen, Dunson, and Carin 2011; Li and McCallum 2006).”
Reducing the sampling complexity of topic models (2014, background): “Likewise, whenever z indicates a term in a hierarchy, it leads to structured and mixed-content annotations [19, 2, 4, 12].”
Beta-Negative Binomial Process and Poisson Factor Analysis (2011, methodology): “It was introduced for machine learning by Thibaux and Jordan (2007) and Jordan (2010).”
Modeling Documents with Deep Boltzmann Machines (2013, methodology): “Directed graphical models, such as Latent Dirichlet Allocation (LDA), CTM, H-LDA, have been extensively used for this [3, 2, 8].”
Network Flow Algorithms for Structured Sparsity (2010, neutral): “Previously approached using non-parametric Bayesian methods (Hierarchical Chinese Restaurant Process and nested Dirichlet Process): [Blei et al., 2010] Can we achieve similar performance with simple matrix factorization formulation?”
Hierarchical Semi-supervised Classification with Incomplete Class Hierarchies (2016, background): “Another set of techniques focus on completely unsupervised information extraction and ontology discovery [15, 4, 21, 10].”
Distance dependent Chinese restaurant processes (2009, positive): “That said, CRP-based methods have been extensively applied to text modeling and natural language processing (Teh et al., 2006; Johnson et al., 2007; Li et al., 2007; Blei et al., 2010).”
Tree-Structured Stick Breaking for Hierarchical Data (2010, methodology): “The nested Chinese restaurant process (nCRP) [17] provides a distribution over trees of unbounded width and depth, but data correspond to paths of infinite length, requiring an additional distribution over depths that is not path-dependent.”
Proximal Methods for Hierarchical Sparse Coding (2010, neutral): “Let us consider an input signal of dimension m , typically an image described by its m pixels, which we represent by a vector x in R m .”
Nonparametric Method for Data-driven Image Captioning (2014, methodology): “For example, previous work in nonparametric hierarchical topic modeling (Blei et al., 2010) and scene labeling (Liu et al., 2011) may provide avenues for further improvement of this model.”
Knowledge Graph Embedding: A Survey of Approaches and Applications (2017, methodology): “These semantic components can be learned automatically from the data using the Chinese restaurant process [60], [61].”
Statistical topic models for multi-label document classification (2011, background): “ several models that represent or induce topic dependencies have been investigated in the past for unsupervised topic modeling (e.g., Blei and Lafferty (2005); Teh et al. (2004); Mimno et al. (2007); Blei et al. (2010)). Although these models are related to varying degrees to the DependencyLDA model”
iBCM: Interactive Bayesian Case Model Empowering Humans via Intuitive Interaction (2015, background): “(This assumption can easily be relaxed through extension to a non-parametric mixture model, such as [8].”