Alleviating Semantics Distortion in Unsupervised Low-Level Image-to-Image Translation via Structure Consistency Constraint

By Jiaxian Guo, Jiacheng Li, Huan Fu, Mingming Gong, Kun Zhang, Dacheng Tao. 2022

24 citations recorded by the source corpus.

Unsupervised image-to-image (I21) translation aims to learn a domain mapping function that can preserve the semantics of the input images without paired data. However, because the underlying semantics distributions in the source and target domains are often mismatched, current distribution matching-based methods may distort the semantics when matching distributions, resulting in the inconsistency between the input and translated images, which is known as the semantics distortion problem. In this paper, we focus on the low-level I21 translation, where the structure of images is highly related to their semantics. To alleviate semantic distortions in such translation tasks without paired supervision, we propose a novel I21 translation constraint, called Structure Consistency Constraint (SCC), to promote the consistency of image structures by reducing the randomness of color transformation in the translation process. To facilitate estimation and maximization of SCC, we propose an approximate representation of mutual information called relative Squared-loss Mutual Information (rSMI) that enjoys efficient analytic solutions. Our SCC can be easily incorporated into most existing translation models. Quantitative and qualitative comparisons on a range of low-level I21 translation tasks show that translation models with SCC outperform the original models by a significant margin with little additional computational and memory costs.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Liu Liu
2. Yajing Kong
3. Dacheng Tao
4. Zhen Wang
5. Jiaxian Guo
6. Jun Zhu
7. Hang Su
8. Liyuan Wang

In Computer Vision and Pattern Recognition

1. Liu Liu
2. Dacheng Tao
3. Zhen Wang
4. Yajing Kong
5. Jiaxian Guo
6. Luc Van Gool
7. Ming-Hsuan Yang
8. Changxing Ding

Citations in the corpus

• Online Continual Learning with Contrastive Vision Transformer (2022, positive): “Inspired by recent breakthroughs in self-supervised learning [16,11,30,72], we find that the knowledge learned by supervised contrastive learning [31,41,28] reveals greater robustness and transferability.”