DISCO Nets : DISsimilarity COefficients Networks

By Diane Bouchacourt, Pawan Kumar Mudigonda, Sebastian Nowozin. 2016

65 citations recorded by the source corpus.

We present a new type of probabilistic model which we call DISsimilarity COefficient Networks (DISCO Nets). DISCO Nets allow us to efficiently sample from a posterior distribution parametrised by a neural network. During training, DISCO Nets are learned by minimising the dissimilarity coefficient between the true distribution and the estimated distribution. This allows us to tailor the training to the loss related to the task at hand. We empirically show that (i) by modeling uncertainty on the output value, DISCO Nets outperform equivalent non-probabilistic predictive networks and (ii) DISCO Nets accurately model the uncertainty of the output, outperforming existing probabilistic models based on deep neural networks.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Model Architectures

Citations in the corpus

Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning (2017, background): “The Deep Learning community has recently proposed Generative Adversarial Networks (GANs) [29, 68, 70], which are still being intensively developed [3, 9, 31, 41, 54].”
Kernel Mean Embedding of Distributions: A Review and Beyonds (2016, background): “…one can generate samples by feeding the models with random noise Z. Generative Adversarial Networks (GAN) proposed by Goodfellow et al. (2014) is one of the most popular deep generative models used in several computer vision applications, see, e.g., Bouchacourt et al. (2016) and references therein.”
A Spectral Energy Distance for Parallel Speech Synthesis (2020, positive): “(4) Loss functions of this type were used by [8, 23] and [4] to train generative models without requiring a tractable likelihood function.”
Lightweight Probabilistic Deep Networks (2018, neutral): “[5, 6].”
Multi-Level Variational Autoencoder: Learning Disentangled Representations from Grouped Observations (2017, methodology): “Examples of implicit models include Generative Adversarial Networks (GANs) developed by Goodfellow et al. [2014] and kernel based models, see Li et al. [2015], Dziugaite et al. [2015], Bouchacourt et al. [2016].”
Stabilizing Training of Generative Adversarial Networks through Regularization (2017, methodology): “These include the Wasserstein distance used in Wasserstein GANs (WGAN) Arjovsky & Bottou (2017) as well as RKHS-induced maximum mean discrepancies Gretton et al. (2012); Li et al. (2015); Bouchacourt et al. (2016), which all remain well-deﬁned.”
Dissimilarity Coefficient Based Weakly Supervised Object Detection (2018, unknown): “(14) The above objective function is similar to the one used in [4] for supervised learning of Discrete DISCO Nets.”
Cross-Modal Deep Variational Hand Pose Estimation (2018, unknown): “In [2], VAE is also deployed for depth based hand pose estimation.”
Geometrical Insights for Implicit Generative Modeling (2017, methodology): “1) and more direct learning algorithms [17, 33, 11] ? The most impressive achievement associated with the implicit modeling approach certainly is the generation of photo-realistic random images that resemble the images provided as training data [15, 41, 25].”
Demystifying MMD GANs (2018, methodology): “If we take an expectation over Y ∼ Q , we recover the scoring rule optimized by the DISCO Nets algorithm (Bouchacourt et al., 2016, Section 3.3).”
A Probabilistic U-Net for Segmentation of Ambiguous Images (2018, background): “Beside that, our design choice of late injection has the additional advantage that we can produce a large set of samples for a given image at a very low computational cost: for each new sample from the latent space only the network part after the injection needs to be re-executed to produce the corr”
Adversarial Message Passing For Graphical Models (2016, background): “[26] Diane Bouchacourt, M Pawan Kumar, and Sebastian Nowozin.”
Single-Model Uncertainties for Deep Learning (2018, background): “Some of these models are conditional generative adversarial networks [54] and DiscoNets [7].”
V2V-PoseNet: Voxel-to-Voxel Prediction Network for Accurate 3D Hand and Human Pose Estimation from a Single Depth Map (2017, methodology): “Most of the existing methods for 3D pose estimation from a single depth map [1, 2, 6, 13–15, 23, 28–30, 46] are based on the model in Figure 2(a) that takes a 2D depth image and directly regresses 3D coordinates.”