Understanding the Limitations of Conditional Generative Models

By Ethan Fetaya, Jörn-Henrik Jacobsen, Will Grathwohl, Richard S. Zemel. 2019

65 citations recorded by the source corpus.

Class-conditional generative models hold promise to overcome the shortcomings of their discriminative counterparts. They are a natural choice to solve discriminative tasks in a robust manner as they jointly optimize for predictive performance and accurate modeling of the input distribution. In this work, we investigate robust classification with likelihood-based generative models from a theoretical and practical perspective to investigate if they can deliver on their promises. Our analysis focuses on a spectrum of robustness properties: (1) Detection of worst-case outliers in the form of adversarial examples; (2) Detection of average-case outliers in the form of ambiguous inputs and (3) Detection of incorrectly labeled in-distribution inputs. Our theoretical result reveals that it is impossible to guarantee detectability of adversarially-perturbed inputs even for near-optimal generative classifiers. Experimentally, we find that while we are able to train robust models for MNIST, robustness completely breaks down on CIFAR10. We relate this failure to various undesirable model properties that can be traced to the maximum likelihood training objective. Despite being a common choice in the literature, our results indicate that likelihood-based conditional generative models may are surprisingly ineffective for robust classification.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

Shortcut learning in deep neural networks (2020, background): “By itself, however, this does not necessarily lead to representations useful for downstream tasks [140], let alone outof-distribution generalisation.”
ECGTwin: Personalized ECG Generation Using Controllable Diffusion Model (2025, unknown): “…the abundance of conditions offers the potential for fine-grained control, it also poses a significant challenge: if not properly integrated, these conditions may confuse the generative model, leading to degraded performance or even model collapse (Fetaya et al. 2020; Thanh-Tung and Tran 2020).”