Learning nonlinear constraints with contrastive backpropagation

By Andriy Mnih, Geoffrey E. Hinton. 2005

31 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Molecular Biology, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

In Optimization & Training

Citations in the corpus

VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Models (2020, methodology): “Early variants of EBMs include models whose energy is defined over both data and auxiliary latent variables (Salakhutdinov & Hinton, 2009; Hinton, 2012), and models using only data variables (Hinton, 2002; Mnih & Hinton, 2005).”
Implicit Generation and Generalization in Energy-Based Models (2019, background): “In contrast(Mnih and Hinton, 2004; Hin-ton et al., 2006) proposed EBMs where inputs are directly mapped to outputs, a structure we follow.”
Learning Deep Energy Models (2011, methodology): “Our work builds on Hinton et al. (2006b), which introduced the contrastive backpropagation algorithm, and also the work of Mnih & Hinton (2005), who first showed how to stack a product of Student-t model.”
How to Train Your Energy-Based Model for Regression (2020, background): “Energy-based models (EBMs) [27] have a rich history in machine learning [2, 16, 34, 41, 49].”
Energy-Based Models for Deep Probabilistic Regression (2020, methodology): “Energy-Based Models Our approach is of course also related to the theoretical framework of energy-based models, which often has been employed for machine learning problems in the past [35, 20, 30].”
End-to-End Learning for Structured Prediction Energy Networks (2017, methodology): “In contrastive backprop , this is circumvented by performing contrastive divergence training, with Hamiltonian Monte Carlo sampling from the energy surface (Mnih & Hinton, 2005; Hinton et al., 2006; Ngiam et al., 2011).”
Implicit Generation and Modeling with Energy Based Models (2019, background): “Compositionality: If we think of energy functions as costs for a certain goals or constraints, summation of two or more energies corresponds to satisfying all their goals or constraints [Mnih and Hinton, 2004, Haarnoja et al., 2017].”
Unsupervised Learning of Visual Representations using Videos (2015, methodology): “This approach was shown to work by Mnih and Hinton (2005) for modeling the coordinates of a moving arm containing several segments.”
Compositional Visual Generation with Energy Based Models (2020, background): “Such an approach allows viewing factors of variation as constraints [19].”
Efficient Learning of Generative Models via Finite-Difference Score Matching (2020, background): “EBMs allow more flexible architectures [9, 11] with simpler compositionality [15, 41] compared to other explicit generative models [46, 16], and have better stability and mode coverage in training [30, 31, 71] compared to implicit generative models [10].”