MD-GAN: Multi-Discriminator Generative Adversarial Networks for Distributed Datasets

By Corentin Hardy, Erwan Le Merrer, Bruno Sericola. 2018

226 citations recorded by the source corpus.

A recent technical breakthrough in the domain of machine learning is the discovery and the multiple applications of Generative Adversarial Networks (GANs). Those generative models are computationally demanding, as a GAN is composed of two deep neural networks, and because it trains on large datasets. A GAN is generally trained on a single server. In this paper, we address the problem of distributing GANs so that they are able to train over datasets that are spread on multiple workers. MD-GAN is exposed as the first solution for this problem: we propose a novel learning procedure for GANs so that they fit this distributed setup. We then compare the performance of MD-GAN to an adapted version of federated learning to GANs, using the MNIST, CIFAR10 and CelebA datasets. MD-GAN exhibits a reduction by a factor of two of the learning complexity on each worker node, while providing better or identical performances with the adaptation of federated learning. We finally discuss the practical implications of distributing GANs.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

FedML: A Research Library and Benchmark for Federated Machine Learning (2020, methodology): “In terms of training procedures, the training procedures in federated GAN [59, 60] and transfer learning-based FL [61, 62, 63, 64, 65] are significantly different from the vanilla FedAvg algorithm [66].”
Generative Models for Effective ML on Private, Decentralized Datasets (2019, background): “An example is training GANs (Hardy et al., 2018) on decentralized data across multiple data centers.”
Wireless Network Intelligence at the Edge (2018, background): “Toward this vision, at the physical layer, an E2E communication framework was studied under unknown channels using an autoencoder (AE) [19], [20], recurrent NN (RNN) [21], and generative adversarial network (GAN) [22].”
Federated Generative Privacy (2019, methodology): “We compare learning performance with the baseline centralised model trained on original data, as well as the same model trained on artificial samples obtained from the centrally trained GAN (CentGP) and from MD-GAN [10].”
Hausdorff GAN: Improving GAN Generation Quality With Hausdorff Metric (2021, methodology): “To validate this, we utilize the MNIST Score [20], which is similar to the Inceptions Score [5] but using a classiﬁer adapted to the MNIST training data instead of the Inception network, to evaluate the quality of generated data.”
Private FL-GAN: Differential Privacy Synthetic Data Generation Based on Federated Learning (2020, methodology): “In addition, in order to optimize the training of GAN models in distributed datasets, [17] proposed a method called MD-GAN.”
Generative Adversarial Networks in Time Series: A Systematic Literature Review (2022, background): “Decentralised/distributed approaches to GAN algorithms require large communication bandwidth to ensure convergence [5, 46] and are also subject to strict privacy constraints.”