Unifying Non-Maximum Likelihood Learning Objectives with Minimum KL Contraction
By Siwei Lyu. 2011
21 citations recorded by the source corpus.
Subfields: Optimization & Training
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Optimization & Training
- Jascha Narain Sohl-Dickstein
- Surya Ganguli
- Josh Tenenbaum
- Balaji Lakshminarayanan
- Yilun Du
- Niru Maheswaranathan
- Shakir Mohamed
- Conor Durkan
Citations in the corpus
- Deep Unsupervised Learning using Nonequilibrium Thermodynamics (2015, background): “…instance mean field theory and its expansions [39, 40], variational Bayes [19], contrastive divergence [44, 13], minimum probability flow [36, 35], minimum KL contraction [28], proper scoring rules [9, 32], score matching [15], pseudolikelihood [5], loopy belief propagation [29], and many, many more.”
- Learning in Implicit Generative Models (2016, methodology): “This piques our interest in more general approaches for non-maximum likelihood and likelihood-free estimation methods, of which there is much work (Frogner et al., 2015; Gutmann and Hyvärinen, 2012; Hall, 2005; Lyu, 2011; Marin et al., 2012).”
- Improved Contrastive Divergence Training of Energy Based Models (2020, background): “If Π represents a MCMC transition kernel, this property is guaranteed (Lyu, 2011).”
- On Maximum Likelihood Training of Score-Based Generative Models (2021, background): “A wide variety of related approaches are covered by Lyu [2011], where learning involves minimizing the change in KL divergence after application of a ‘KL contraction operator’.”
- How to Train Your Energy-Based Models (2021, background): “Typical examples of this strategy include minimum velocity learning (Movellan, 2008; Wang et al., 2020), minimum probability flow (Sohl-Dickstein et al., 2011) and minimum KL contraction (Lyu, 2011).”
- Under Review as a Conference Paper at Iclr 2017 Learning in Implicit Generative Models (2017, methodology): “This piques our interest in more general approaches for non-maximum likelihood and likelihood-free estimation methods, of which there is much work (Lyu, 2011; Gutmann and Hyvärinen, 2012; Hall, 2005; Marin et al., 2012; Frogner et al., 2015).”