Robust Optimal Transport with Applications in Generative Modeling and Domain Adaptation

By Yogesh Kumar Balaji, Ramalingam Chellappa, Soheil Feizi. 2020

132 citations recorded by the source corpus.

Optimal Transport (OT) distances such as Wasserstein have been used in several areas such as GANs and domain adaptation. OT, however, is very sensitive to outliers (samples with large noise) in the data since in its objective function, every sample, including outliers, is weighed similarly due to the marginal constraints. To remedy this issue, robust formulations of OT with unbalanced marginal constraints have previously been proposed. However, employing these methods in deep learning problems such as GANs and domain adaptation is challenging due to the instability of their dual optimization solvers. In this paper, we resolve these issues by deriving a computationally-efficient dual form of the robust OT optimization that is amenable to modern deep learning applications. We demonstrate the effectiveness of our formulation in two applications of GANs and domain adaptation. Our approach can train state-of-the-art GAN models on noisy datasets corrupted with outlier distributions. In particular, our optimization computes weights for training samples reflecting how difficult it is for those samples to be generated in the model. In domain adaptation, our robust OT formulation leads to improved accuracy compared to the standard adversarial adaptation methods. Our code is available at this https URL.

Subfields: Artificial Intelligence, Civil and Structural Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Li Shen
2. Mingchen Gao
3. Zhenyi Wang
4. Qiuling Suo
5. Le Fang
6. Tiehang Duan
7. Dongling Zhan

In Computer Vision and Pattern Recognition

1. Li Shen
2. Mingchen Gao
3. Zhenyi Wang
4. Qiuling Suo
5. Le Fang
6. Tiehang Duan

Citations in the corpus

• Meta-Learning with Less Forgetting on Large-Scale Non-Stationary Task Distributions (2022, background): “Given two discrete distributions, μ = ∑n i=1 uiδei and ν = ∑m j=1 vjδgj , respectively, the OT distance [8] between μ and ν is defined as the optimal value of the following problem:”
• Co-Transport for Class-Incremental Learning (2021, neutral): “model fusion [37], domain adaptation [7], object detection [10], model reuse [56, 57], and generative models [2, 4].”