Distribution Matching Losses Can Hallucinate Features in Medical Image Translation

By Joseph Paul Cohen, Margaux Luck, Sina Honari. International Conference on Medical Image Computing and Computer-Assisted Intervention, 2018

352 citations recorded by the source corpus.

This paper discusses how distribution matching losses, such as those used in CycleGAN, when used to synthesize medical images can lead to mis-diagnosis of medical conditions. It seems appealing to use these new image synthesis methods for translating images from a source to a target domain because they can produce high quality images and some even do not require paired data. However, the basis of how these image translation models work is through matching the translation output to the distribution of the target domain. This can cause an issue when the data provided in the target domain has an over or under representation of some classes (e.g. healthy or sick). When the output of an algorithm is a transformed image there are uncertainties whether all known and unknown class labels have been preserved or changed. Therefore, we recommend that these translated images should not be used for direct interpretation (e.g. by doctors) because they may lead to misdiagnosis of patients based on hallucinated image features by an algorithm that matches a distribution. However there are many recent papers that seem as though this is the goal.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/c42faad6f2ab45bb3a1cf27408cae32f575632b6

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Self-derived organ attention for unpaired CT-MRI deep domain adaptation based MRI segmentation (2020, methodology): “CycleGAN is frequently used for cross-domain adaptation (Cohen et al 2018, Huo et al 2018, Liu 2019), because the cyclical consistency loss used in this network ensures the preservation of global image statistics.”
GANs for Medical Image Analysis (2018, background): “A major limitation of GANs, as pointed out by [32], is the lack of a guarantee that tumors/lesions are preserved during image-to-image translation.”
A Gentle Introduction to Deep Learning in Medical Image Processing (2018, background): “Cohen et al. demonstrate even worse eﬀects [102].”
Generative Adversarial Network in Medical Imaging: A Review (2018, background): “(2015) HVSMR 2016 whole heart and great vessel segmentation Heart, Vessel MR D8 IXI Analysis of brain development Brain MR D9 DSB2015 End-systolic/diastolic volumes measurement Heart MR D10 Mridata MRI Reconstruction Knee MR D11 Crimi et al. (2016) BrainLes Lesion segmentation Brain MR D12 ADNI Alzh”
Benchmarking Bias Mitigation Algorithms in Representation Learning through Fairness Metrics (2021, background): “In particular, [1] outline the biases exhibited by learning algorithms whose goal is to match the data distribution in an adversarial setting.”
COVID-19 Image Data Collection: Prospective Predictions Are the Future (2020, methodology): “Out-of-distribution (OoD) tools [Shafaei et al., 2018] or unpaired distribution matching models [Zhu et al., 2017; Kim et al., 2017] could capture the shift in distributions and present them as changes to images Cohen et al. [2018].”