Wasserstein Training of Restricted Boltzmann Machines

By Grgoire Montavon, K. Müller, Marco Cuturi. 2016

116 citations recorded by the source corpus.

Subfields: Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Probabilistic & Bayesian Methods

Citations in the corpus

Wasserstein GAN (2017, methodology): “On another great line of research, the recent work of [14] has explored the use of Wasserstein distances in the context of learning for Restricted Boltzmann Machines for discrete spaces.”
Wireless Network Intelligence at the Edge (2018, methodology): “In this regard, the local and global models defined for FL require to compute the gradient as follows [210]:”
The Cramer Distance as a Solution to Biased Wasserstein Gradients (2017, background): “For example, Frogner et al. (2015) and Montavon et al. (2016) reported the need for a mixed KL-Wasserstein loss to obtain good empirical results, with the latter explicitly discussing the issue of wrong minima when using Wasserstein gradients.”
Learning Generative Models with Sinkhorn Divergences (2017, methodology): “To remedy these issues, and in line with several recent proposals [2, 26, 4, 1], we propose to shift away from information divergence based methods (among which the MLE) and consider instead the geometry of optimal transport [36, 31] to define such a fitting criterion.”
Computational Optimal Transport (2018, methodology): “It has been used in the context of generative models by [Montavon et al., 2016] to train restricted Boltzmann machines and in [Bernton et al.”
Adversarial Variational Optimization of Non-Differentiable Simulators (2017, methodology): “1 points out that (Montavon et al., 2016) proposed an approximation of the gradient of an entropy-regularized Wasserstein distance, which uses a similar duality.”
Earth Mover’s Distance Minimization for Unsupervised Bilingual Lexicon Induction (2017, background): “Apart from the usage as a regularizer (Zhang et al., 2016a), the EMD can also play other roles in optimization programs designed for various applications (Cuturi and Doucet, 2014; Frogner et al., 2015; Montavon et al., 2016).”
Interpolating between Optimal Transport and MMD using Sinkhorn Divergences (2018, background): “…from computer graphics (Bonneel et al., 2016) to supervised learning (Frogner et al., 2015), unsupervised density ﬁtting (Bassetti et al., 2006) and generative model learning (Montavon et al., 2016; Arjovsky et al., 2017; Salimans et al., 2018; Genevay et al., 2018; Sanjabi et al., 2018).”
GAN and VAE from an Optimal Transport Point of View (2017, methodology): “The MKE approach can be used directly in the case where ( µ θ ) θ is a statistical model, namely a parameterized family of probability distributions with a given density with respect to a dominant base measure, as considered for instance with exponential families on discrete spaces in [11].”
Stochastic Optimization for Large-scale Optimal Transport (2016, background): “Solving these semi-discrete problems efficiently could have a significant impact for applications to density fitting with an OT loss [2] for machine learning applications, see [13].”
Sliced Wasserstein Distance for Learning Gaussian Mixture Models (2017, methodology): “It has been shown that optimizing with respect to the Wasserstein loss has various practical benefits over the KL-divergence loss [43, 15, 38, 3, 19].”
Wasserstein Generative Adversarial Networks (2017, unknown): “On another great line of research, the recent work of (Montavon et al., 2016) has explored the use of Wasserstein distances in the context of learning for Restricted Boltzmann Machines for discrete spaces.”
Generalized Sliced Wasserstein Distances (2019, background): “…and machine learning due to its convenient theoretical properties and applications on many domains (Solomon et al., 2014; Frogner et al., 2015; Montavon et al., 2016; Kolouri et al., 2017; Courty et al., 2017; Peyr´e & Cu-turi, 2018; Schmitz et al., 2018), especially in implicit generative…”