Discriminator optimal transport

By Akinori Tanaka. 2019

55 citations recorded by the source corpus.

Within a broad class of generative adversarial networks, we show that discriminator optimization process increases a lower bound of the dual cost function for the Wasserstein distance between the target distribution $p$ and the generator distribution $p_G$. It implies that the trained discriminator can approximate optimal transport (OT) from $p_G$ to $p$. Based on some experiments and a bit of OT theory, we propose discriminator optimal transport (DOT) scheme to improve generated images. We show that it improves inception score and FID calculated by un-conditional GAN trained by CIFAR-10, STL-10 and a public pre-trained model of conditional GAN trained by ImageNet.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Statistical and Nonlinear Physics, Mathematical Optimization

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Statistical and Nonlinear Physics

In Mathematical Optimization

Citations in the corpus

Your GAN is Secretly an Energy-based Model and You Should use Discriminator Driven Latent Sampling (2020, methodology): “We use the implementation from scikit-learn, and rescale the coordinates as suggested by [12].”
Score-based Generative Modeling in Latent Space (2021, neutral): “Our work also bears a resemblance to recent methods on improving the sampling quality in generative adversarial networks using gradient flows in the latent space [65, 66, 67, 68], with the main difference that these prior works use a discriminator to update the latent variables, whereas we train an ”
LOGAN: Latent Optimisation for Generative Adversarial Networks (2019, background): “(2019) suggested this term is related to optimal transport; more recently, Tanaka (2019) formalised this connection in Discriminator Optimal Transport (DOT).”
Freeze the Discriminator: a Simple Baseline for Fine-Tuning GANs (2020, methodology): “…[24, 47, 42], using an auxiliary task to facilitate training [7, 26, 48, 49], improving an inference procedure of suboptimal models [2, 39, 29, 38], using an expressive prior distribution [13], actively choosing samples to give supervision for conditional generation [29], or actively sampling…”
OT-Flow: Fast and Accurate Continuous Normalizing Flows via Optimal Transport (2020, methodology): “OT is also used in other generative models (Sanjabi et al. 2018; Salimans et al. 2018; Lei et al. 2019; Lin, Lensink, and Haber 2019; Avraham, Zuo, and Drummond 2019; Tanaka 2019).”
Refining Deep Generative Models via Discriminator Gradient Flow (2020, methodology): “Hyperparameters We ran DOT for 100 steps and performed gradient descent using the Adam optimizer with a learning rate of 0.01 and β = (0., 0.9) as suggested by Tanaka (2019).”
Discriminator Contrastive Divergence: Semi-Amortized Generative Modeling by Exploring Energy of the Discriminator (2020, methodology): “[33] proposes discriminator optimal transport (DOT) to leverage the optimal transport plan implied by WGANs’ discriminator, which is orthogonal to our method.”
Generalized Energy Based Models (2020, methodology): “Sampling: For (DOT) Tanaka (2019), we use the following objective: z 7→‖z−zy+ ‖+ 1 keff E◦G(z) (31) where zy is sampled from a standard Gaussian, is a perturbation meant to stabilize sampling and keff is the estimated Lipschitz constant of E ◦B. Note that (31) uses a flipped sign for the E ◦B…”
KALE: When Energy-Based Learning Meets Adversarial Training (2020, background): “While there has been a recent interest in using the discriminator/energy to improve the quality of the generator/base (Azadi et al., 2019; Grover et al., 2019; Turner et al., 2019; Tanaka, 2019) (Section 5), our approach emerges naturally from the energy model we consider.”
Exponential Tilting of Generative Models: Improving Sample Quality by Training and Sampling from Latent Energy (2020, methodology): “Recently, there is a line of literature that improves the sample quality of pre-trained GANs [23, 5].”