Minimax Statistical Learning and Domain Adaptation with Wasserstein Distances

By Jaeho Lee, Maxim Raginsky. 2017

21 citations recorded by the source corpus.

As opposed to standard empirical risk minimization (ERM), distributionally robust optimization aims to minimize the worst-case risk over a larger ambiguity set containing the original empirical distribution of the training data. In this work, we describe a minimax framework for statistical learning with ambiguity sets given by balls in Wasserstein space. In particular, we prove a generalization bound that involves the covering number properties of the original ERM problem. As an illustrative example, we provide generalization guarantees for domain adaptation problems where the Wasserstein distance between the source and target domain distributions can be reliably estimated from unlabeled samples.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

Agnostic Federated Learning (2019, methodology): “Our optimization solution also differs from the work of Farnia and Tse (2016) and Lee and Raginsky (2017) on local minimax results, where samples are drawn from a single source D , and where the generalization error is minimized over a set of locally ambiguous distributions ̂ D , where ̂ D is the…”
Certifying Some Distributional Robustness with Principled Adversarial Training (2017, background): “One natural application is domain adaptation [41]; Lee and Raginsky provide guarantees similar to ours for the empirical minimizer of the robust saddle-point problem (1) and give specialized bounds for domain adaptation problems.”
Generalizing to Unseen Domains via Adversarial Data Augmentation (2018, background): “The literature on adversarial training [10, 34, 20, 12] is closely related to our work, since the main goal is to devise training procedures that learn models robust to fluctuations in the input.”
Model Vulnerability to Distributional Shifts over Image Transformation Sets (2019, background): “This paradigm has been extensively studied in a substantial body of works [18, 47, 29, 20, 38, 35].”
Regularization via Mass Transportation (2017, methodology): “…bounds for the worst-case prediction loss with respect to a Wasserstein ball centered at the true distribution are derived by Lee and Raginsky [27] in order to address emerging challenges in domain adaptation problems, where the distributions of the training and test samples can diﬀer.”
Learning Models with Uniform Performance via Distributionally Robust Optimization (2018, background): “An alternative to our f -divergence based sets {Q : Df (Q||P0) ≤ ρ} are Wasserstein balls [99, 72, 100, 81, 18, 17, 41, 38, 87, 65].”
A Review of Domain Adaptation without Target Labels (2019, methodology): “Finally, variations on minimax estimators have been proposed in the context of domain-adversarial networks [182], multiple-kernel support vector machines [183] and with using Wasserstein distances [184].”