Information processing in dynamical systems: foundations of harmony theory

By Paul Smolensky. 1986

Subfields: Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Probabilistic & Bayesian Methods

Citations in the corpus

Generative Adversarial Nets (2014, background): “An alternative to directed graphical models with latent variables are undirected graphical models with latent variables, such as restricted Boltzmann machines (RBMs) [27, 16], deep Boltzmann machines (DBMs) [26] and their numerous variants.”
Pylearn2: a machine learning research library (2013, methodology): “Pylearn2 has implementations of several models that were developed at LISA, including denoising auto-encoders (DAEs) [53], contractive auto-encoders (CAEs) [42] including higher-order CAEs [43], spike-and-slab RBMs (ssRBMs) [11] including ssRBMs with pooling [12], reconstruction sampling autoencoder”
Music Transformer: Generating Music with Long-Term Structure (2018, background): “Alternatively, a 2D pianoroll-like representation (see A.1 for more details) can be decomposed into a sequence of multi-hot pitch vectors, and their joint probability distributions can be captured using Restricted Boltzmann Machines (Smolensky, 1986; Hinton et al., 2006) or Neural Autoregressive Dis”
Neural Autoregressive Distribution Estimation (2016, background): “These include the restricted Boltzmann machine (Smolensky, 1986; Freund and Haussler, 1992) and its multilayer extension, the deep Boltzmann machine (Salakhut-dinov and Hinton, 2009), which dominate the literature on undirected neural networks.”
Contrastive Multiview Coding (2019, methodology): “ve of downstream tasks such as object recognition. Representations better suited to such tasks have been learnt using deep neural networks, starting with seminal techniques such as Boltzmann machines [55, 50], autoencoders [23], variational autoencoders [30], generative adversarial networks [20] and”
Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles (2016, methodology): “A family of popular probabilistic models is the Restricted Boltzmann Machine (RBM) [32,16], which makes training tractable by imposing a bipartite graph between latent and observed variables.”
Energy-Inspired Models: Learning with Sampler-Induced Distributions (2019, methodology): “Different choices of energy function recover well-known probabilistic models including Markov random ﬁelds [35], (restricted) Boltzmann machines [62, 24, 29], and conditional random ﬁelds [40].”
Density estimation using Real NVP (2016, neutral): “Occurrences of such models include probabilistic undirected graphs such as Restricted Boltzmann Machines [51] and Deep Boltzmann Machines [46].”
Generating Images with Perceptual Similarity Metrics based on Deep Networks (2016, background): “A prominent class of probabilistic models of images are restricted Boltzmann machines (Hinton & Sejnowski, 1986; Smolensky, 1987; Hinton & Salakhutdinov, 2006) and their deep variants (Hinton et al.”
Generative adversarial networks (2014, neutral): “ins are necessary. 2 Related work An alternative to directed graphical models with latent variables are undirected graphical models with latent variables, such as restricted Boltzmann machines (RBMs) [27, 16], deep Boltzmann machines (DBMs) [26] and their numerous variants. The interactions within s”
Representation Learning: A Review and New Perspectives (2012, background): “The RBM is likely the most popular subclass of Boltzmann machine (Smolensky, 1986).”
Importance Weighted Autoencoders (2015, background): “Some models are deﬁned in terms of Boltzmann distributions (Smolensky, 1986; Salakhutdinov & E., 2009).”
Generating Images from Captions with Attention (2015, background): “Most of the previous work in generative models has focused on variants of Boltzmann Machines (Smolensky, 1986; Salakhutdi-nov & Hinton, 2009) and Deep Belief Networks (Hinton et al., 2006).”
Modeling Temporal Dependencies in High-Dimensional Sequences: Application to Polyphonic Music Generation and Transcription (2012, background): “This dif-ﬁculty motivates energy-based models which allow us to express the negative log-likelihood of a given conﬁg-uration by an arbitrary energy function, among which the restricted Boltzmann machine (RBM) (Smolensky, 1986) has become notorious.”
Classification Accuracy Score for Conditional Generative Models (2019, background): “The most popular seem to be: • (Inexact) Likelihood models using variational inference (e.g., VAE [17, 18]) • Likelihood using autoregressive models (e.g., PixelCNN [19]) • Likelihood models based on bijections (e.g., GLOW [20], rNVP [21]) • (Possibly inexact) likelihood using energy-based models (e”
Attribute2Image: Conditional Image Generation from Visual Attributes (2015, background): “As a representative example, the restricted Boltzmann machine (RBM) [33] and its variants [28, 25, 30, 15, 37, 31] have been widely studied for image modeling.”
Learning Deep Structured Models (2014, background): “A few attempts to incorporate structure in deep networks have been proposed, restricted Boltzmann machines (RBMs) [9, 24] and its variants [22] are the most common examples.”
Deep learning in neural networks: An overview (2014, neutral): “The Deep Belief Network (DBN) is a stack of Restricted Boltzmann Machines (RBMs) (Smolensky, 1986), which in turn are Boltzmann Machines (BMs) (Hinton & Sejnowski, 1986) with a single layer of feature-detecting units; compare also Higher-Order BMs (Memisevic & Hinton, 2010).”
Recent Progress on Generative Adversarial Networks (GANs): A Survey (2019, methodology): “Restricted Boltzmann Machines [4] and its extension models (e.g., Deep Belief Networks (DBNs) [5], Deep Boltzmann Machinescite [6]) are always based on maximum likelihood estimation.”