VisDA: A Synthetic-to-Real Benchmark for Visual Domain Adaptation

By Xingchao Peng, Ben Usman, Neela Kaushik, Dequan Wang, Judy Hoffman, Kate Saenko. 2018

206 citations recorded by the source corpus.

The success of machine learning methods on visual recognition tasks is highly dependent on access to large labeled datasets. However, real training images are expensive to collect and annotate for both computer vision and robotic applications. The synthetic images are easy to generate but model performance often drops significantly on data from a new deployment domain, a problem known as dataset shift, or dataset bias. Changes in the visual domain can include lighting, camera pose and background variation, as well as general changes in how the image data is collected. While this problem has been studied extensively in the domain adaptation literature, progress has been limited by the lack of large-scale challenge benchmarks.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

Citations in the corpus

Break-It-Fix-It: Unsupervised Learning for Program Repair (2021, background): “…Kamath et al., 2020), as well as from synthetic data to real data (including sim2real) (Wang et al., 2015; Ganin & Lempitsky, 2015; Richter et al., 2016; Peng et al., 2018; Hellendoorn et al., 2019; Xu et al., 2020; Bellemare et al., 2020), as synthetic data can be easier to obtain than real data.”
Large image datasets: A pyrrhic win for computer vision? (2020, background): “This is a nascent ﬁeld with some promising results emerging in unsupervised domain adaptation across visual domains [61] and universal digit classiﬁcation [64].”
WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020, neutral): “In particular, synthetic benchmarks that study transfers from synthetic to real data (Ganin and Lempitsky, 2015; Richter et al., 2016; Peng et al., 2018) can be important tools for tackling real-world problems: even though the data is synthesized and by definition, not real, the synthetic-to-real…”
LoveDA: A Remote Sensing Land-Cover Dataset for Domain Adaptive Semantic Segmentation (2021, background): “The Oracle setting is designed to test the upper limit of accuracy in a single domain [31].”
Supplementary : Learning to Detect Open Classes for Universal Domain Adaptation (2020, methodology): “VisDA [9] is a simulation-to-real dataset containing over 280K images across 12 classes.”
DLOW: Domain Flow for Adaptation and Generalization (2018, background): “Recently, there is an increasing interest to apply domain adaptation techniques for semantic segmentation from synthetic data to the real scenario [22, 21, 7, 67, 40, 25, 11, 46, 51, 53, 23, 47, 62, 54, 43, 50, 52, 66, 5].”
Universal Domain Adaptation (2019, background): “VisDA2017 [29] dataset focuses on a special domain adaptation setting (simulation to real).”
Confidence Regularized Self-Training (2019, methodology): “Adaptation for image classification: We consider two adaptation benchmarks: 1) VisDA17 [42] and 2) Office-31 [48].”