Pros and Cons of GAN Evaluation Measures: New Developments
By Ali Borji. Computer Vision and Image Understanding, 2021
380 citations recorded by the source corpus.
This work is an update of a previous paper on the same topic published a few years ago. With the dramatic progress in generative modeling, a suite of new quantitative and qualitative techniques to evaluate models has emerged. Although some measures such as Inception Score, Frechet Inception Distance, Precision-Recall, and Perceptual Path Length are relatively more popular, GAN evaluation is not a settled issue and there is still room for improvement. Here, I describe new dimensions that are becoming important in assessing models (e.g. bias and fairness) and discuss the connection between GAN evaluation and deepfakes. These are important areas of concern in the machine learning community today and progress in GAN evaluation can help mitigate them.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition
View Paper: https://www.semanticscholar.org/paper/a92316de93a376c8d1b3c74cf19812230fb917cc
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Zhengwei Wang
- Tomás Ward
- Eoin Brophy
- Qingyun She
- Ali Borji
- Tuomas Kynkaanniemi
- Timo Aila
- Miika Aittala
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Ming-Hsuan Yang
- Dacheng Tao
- Antonio Torralba
- Xiaogang Wang
- Ali Borji
- Trevor Darrell
- Chen Change Loy
Citations in the corpus
- The Role of ImageNet Classes in Fréchet Inception Distance (2022, background): “Furthermore, several authors (Karras et al., 2020b; Morozov et al., 2021; Nash et al., 2021; Borji, 2022) observe that there exists a discrepancy in the model ranking between human judgement and FID in non-ImageNet data, and proceed to introduce alternative metrics.”
- Generative Adversarial Networks in Time Series: A Systematic Literature Review (2022, background): “A wide range of evaluation metrics has been proposed to evaluate the performance of GANs [9, 10, 98, 99].”
- A Categorical Archive of ChatGPT Failures (2023, background): “Some works have addressed this issue in the domain of image and video generation [4, 5], but less work have been reported for models that generate text.”