Sum-product networks: A new deep architecture

By Hoifung Poon, Pedro M. Domingos. 2011

824 citations recorded by the source corpus.

The key limiting factor in graphical model inference and learning is the complexity of the partition function. We thus ask the question: what are the most general conditions under which the partition function is tractable? The answer leads to a new kind of deep architecture, which we call sum-product networks (SPNs) and will present in this abstract.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Computer Vision and Pattern Recognition, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

In Computer Vision and Pattern Recognition

In Model Architectures

Citations in the corpus

Deep Generative Stochastic Networks Trainable by Backprop (2013, background): “…may well be other very different ways of evading this problem of intractable marginalization, including approaches such as sum-product networks (Poon & Domingos, 2011), which are based on learning a probability function that has a tractable form by construction and yet is from a ﬂexible enough…”
GSNs : Generative Stochastic Networks (2015, background): “One option is to change the family of functions to guarantee that the likelihood is tractable, e.g. with sum–product networks (Poon & Domingos, 2011).”
Deep learning in neural networks: An overview (2014, background): “…2001), e.g., Baldi and Chauvin (1996), Bengio (1991), Bishop (2006), Bottou (1991), Bourlard andMorgan (1994), Dahl, Yu, Deng, and Acero (2012), Hastie, Tibshirani, and Friedman (2009), Hinton, Deng, et al. (2012), Jordan and Sejnowski (2001), Poon and Domingos (2011) and Wu and Shao (2014).”
Neural-Symbolic Computing: An Effective Methodology for Principled Integration of Machine Learning and Reasoning (2019, methodology): “This intractability problem, for-tunately, can be ameliorated using sum-product approach as has been shown in [38].”
Deep Learning: Methods and Applications (2014, background): “Empirical solutions have been found to mitigate this difficulty as reported in [289].”
Benefits of Depth in Neural Networks (2016, background): “, 2012), sum-product networks (where nodes compute polynomials) (Poon and Domingos, 2011), and boosted decision trees; the full definition of semi-algebraic gates appears in Section 2.”
Input Convex Neural Networks (2016, background): “833.0 ICNN - Gradient Decent 872.0 ICNN - Nonconvex 850.9 Sum-product (Poon & Domingos, 2011) 942 Table 2.”
Accurate and conservative estimates of MRF log-likelihood using reverse annealing (2014, background): “…has led researchers to propose alternative generative models for which the log-likelihood can be computed exactly (Larochelle and Murray, 2011; Poon and Domingos, 2011) or lower bounded (Gregor et al., 2014; Mnih and Gregor, 2014), but RBMs and DBMs remain the state-of-the-art for modeling…”
A tutorial survey of architectures, algorithms, and applications for deep learning (2014, background): “Properties of “completeness” and “consistency” constrain the SPN in a desirable way.”
Hinge-Loss Markov Random Fields and Probabilistic Soft Logic (2015, methodology): “Computing the expectation is intractable, so we use a common approximation (e.g., Collins, 2002; Singla and Domingos, 2005; Poon and Domingos, 2011): the values of the potentials at the most probable setting of y with the current parameters, i.e., a MAP state.”
Bayesian Computation Emerges in Generic Cortical Microcircuits through Spike-Timing-Dependent Plasticity (2013, result): “The hierarchical structure, however, that would result from such deeply organized SEM-modules is more reminiscent of a Deep Sum-Product Network [72], a recently presented new architecture, which has a much simpler learning dynamics but arguably a similar expressive power as DBM.”
Image Modeling with Deep Convolutional Gaussian Mixture Models (2021, background): “, [27]), which are tree structures of arbitrary depth, with leaves that represent tractable distribution families (typically multi-variate normal, binomial or student-t distributions).”
Learning the Structure of Sum-Product Networks (2013, background): “Recently, Poon and Domingos (2011) turned this around by introducing sum-product networks (SPNs), a class of models where adding layers increases expressiveness without losing tractability.”
On the Expressive Power of Deep Architectures (2011, background): “Poon and Domingos (2011) introduced deep sum-product networks as a method to compute partition functions of tractable graphical models.”
On the number of response regions of deep feed forward networks with piece-wise linear activations (2013, background): “Bengio and Delalleau (2011) show that deep sum-product networks (Poon and Domingos, 2011) can use exponentially less nodes to express some families of polynomials compared to the shallow ones.”
On the Expressive Power of Deep Learning: A Tensor Analysis (2015, neutral): “A special case of this are the Sum-Product Networks ( SPNs ) presented in Poon and Domingos (2011).”