A Unified Energy-Based Framework for Unsupervised Learning

By Marc'Aurelio Ranzato, Y-Lan Boureau, Sumit Chopra, Yann LeCun. 2007

145 citations recorded by the source corpus.

Subfields: Generative Models, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Generative Models

In Probabilistic & Bayesian Methods

Citations in the corpus

Residual Energy-Based Models for Text Generation (2020, methodology): “Energy-based models have a long history in machine learning (Hopfield, 1982; Hinton, 2002; LeCun et al., 2006; Ranzato et al., 2007).”
On Reinforcement Learning and Distribution Matching for Fine-Tuning Language Models with no Catastrophic Forgetting (2022, neutral): “Energy-based models for language Energy-based models (EBMs) (Hinton, 2002; LeCun et al., 2006; Ranzato et al., 2007) are a family of models in which learning and inference are done by associating an unnormalized probability with each configuration of observed and latent variables.”
Energy-based Generative Adversarial Network (2016, methodology): “The probabilistic binary discriminator in the original formulation of GAN can be seen as one way among many to deﬁne the contrast function and loss functional, as described in LeCun et al. (2006) for the supervised and weakly supervised settings, and Ranzato et al. (2007) for unsupervised learning.”
A practical tutorial on autoencoders for nonlinear feature fusion: Taxonomy, models, software and guidelines (2018, background): “The most common architecture in unsupervised deep learning is that of the encoder-decoder [14].”
A Distributional Approach to Controlled Text Generation (2020, methodology): “Energy Based Models for Text Energy-Based Models (EBMs) (Hinton, 2002; LeCun et al., 2006; Ranzato et al., 2007) are learning frameworks that attracted a lot of attention several decades ago.18 There has been a recent surge of interest in these types of models across a variety of fields.”
Energy-based Out-of-distribution Detection (2020, background): “Energy-based learning [21, 31, 32] provides a uniﬁed framework for many probabilistic and non-probabilistic approaches to learning.”
Real or Fake? Learning to Discriminate Machine from Human Generated Text (2019, background): “Other approaches instead use implicit negatives, by enforcing global constraints on the energy function, like sparsity of the internal representation (Ranzato et al., 2007), for instance.”
Continual Learning in Neural Networks (2019, methodology): “However, we use an unregularized one-layer undercomplete autoencoder and for these, it has been shown [98] that the mean squared error criterion we use as reconstruction error estimates an energy function that outputs low values for input samples similar to training data, and high energy values othe”
Image denoising: Can plain neural networks compete with BM3D? (2012, methodology): “containing a sparsifying logistic) is used in [19] to denoise image patches.”
Hierarchical Convolutional Deep Learning in Computer Vision (2013, background): “A similar approach to autoencoders for unsupervised learning is called predictive sparse decomposition (PSD) [61,96,97,100].”
Energy-Based Models for Code Generation under Compilability Constraints (2021, methodology): “Energy-based models for text Energy-based models (EBMs) (Hinton, 2002; LeCun et al., 2006; Ranzato et al., 2007) are a family of probabilistic graphical models in which learning and inference are done by associating an unnormalized probability with each configuration of observed and latent variables”
Energy-Based Models for Text (2020, methodology): “On the application side, a natural application of the current formulation of EBMs is real/fake text discrimination.”
Hyperparameters Adaptation for Restricted Boltzmann Machines Based on Free Energy (2016, methodology): “This work was supported by Chinese National 863 Program of Demonstration of Digital Medical Service and Technology in Destined Region (Grant No. 2012- AA02A614).”
Semantic Autoencoder for Zero-Shot Learning (2017, methodology): “In this work, we present a novel approach to zeroshot learning based on the encoder-decoder paradigm [43].”
Residual Energy-Based Models for Text (2021, methodology): “Unfortunately, gradient-based MCMC approaches like Hybrid Monte Carlo (Teh et al., 2003) and Langevyn dynamics (Ranzato et al., 2007; Du and Mordatch, 2019; Xie et al., 2016, 2017, 2019, 2018; Gao et al., 2018; Nijkamp et al., 2019) are not applicable when the input is discrete like in text…”
Foundations and Advances in Deep Learning (2014, methodology): “Salakhutdinov (2009) proposed to use tempered transitions (Neal, 1994), while a similar, but not identical sampling method based on tempered chains called parallel tempering was proposed in Publication III and independently by Desjardins et al.”
Learning multiple layers of representation. (2007, methodology): “Versions of this approach are currently being applied to tasks as diverse as denoising images [36, 37 ] ,r etrieving documents [2,38], extracting optical flow [39] ,p redicting the next word in a sentence [40] and predicting what movies people will like [41] .B engio and Le Cun[42] give further reas”