Unbiased look at dataset bias

By Antonio Torralba, Alexei A. Efros. 2011

2,807 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Evaluation & Benchmarks, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Evaluation & Benchmarks

In Safety, Robustness & Interpretability

Citations in the corpus

ImageNet Large Scale Visual Recognition Challenge (2014, background): “reconstruction and KITTI (Geiger et al., 2013) for computer vision in autonomous driving. These datasets along with ILSVRC help benchmark progress in dierent areas of computer vision. Works such as (Torralba and Efros, 2011) emphasize the importance of examining the bias inherent in any standardized”
Microsoft COCO: Common Objects in Context (2014, background): “Torralba and Efros [42] proposed a metric to measure cross-dataset generalization which computes the ‘per-formance drop’ for models that train on one dataset and test on another.”
High-Resolution Image Synthesis with Latent Diffusion Models (2021, background): “Finally, deep learning modules tend to reproduce or exacerbate biases that are already present in the data [22, 38, 91].”
Pretrained Transformers Improve Out-of-Distribution Robustness (2020, background): “Such train-test mismatches occur because evaluation datasets rarely characterize the entire distribution (Torralba and Efros, 2011), and the test distribution typically drifts over time (Quionero-Candela et al.”
Do Better ImageNet Models Transfer Better? (2018, background): “Is the general enterprise of learning widely-useful features doomed to suffer the same fate as feature engineering? Given differences between datasets [69], it is not entirely surprising that features learned on one dataset benefit from some amount of adaptation when applied to another.”
AI and the Everything in the Whole Wide World Benchmark (2021, background): “As Torralba and Efros demonstrate, images from the same class but different datasets are often distinguishable and embody a very specific style of capturing some segment of the real world, unable to ever get to full coverage of every potentially relevant edge case [Torralba and Efros, 2011].”
Break-It-Fix-It: Unsupervised Learning for Program Repair (2021, background): “Such domain shifts typically occur across related but different datasets (Torralba & Efros, 2011; Fang et al., 2013; Venkateswara et al., 2017; Yu et al., 2018b; 2019; Peng et al., 2019; Kamath et al., 2020), as well as from synthetic data to real data (including sim2real) (Wang et al., 2015; Ganin…”
Shortcut learning in deep neural networks (2020, background): “These so-called dataset biases have long been known to be problematic for machine learning algorithms [18].”
Understanding Dataset Difficulty with V-Usable Information (2021, positive): “Datasets are designed to act as proxies for real-world tasks, yet most bear limited semblance to the tasks they purport to reflect (Torralba & Efros, 2011; Recht et al., 2019).”
Revisiting Unreasonable Effectiveness of Data in Deep Learning Era (2017, background): “Another related work is the empirical study by Torralba and Efros [41] that highlighted the dataset biases in current com-”
Measuring Robustness to Natural Distribution Shifts in Image Classification (2020, background): “Reliable classiﬁcation under distribution shift is still out of reach for current machine learning [65, 68, 91].”
FairFace: Face Attribute Dataset for Balanced Race, Gender, and Age (2019, background): “Various unwanted biases in image datasets can easily occur due to biased selection, capture, and negative sets [60].”
Do Image Classifiers Generalize Across Time? (2019, background): “In a similar vein of study, [29] studies distribution shift across different computer vision data sets such as Caltech-101, PASCAL, and ImageNet.”
Do ImageNet Classifiers Generalize to ImageNet? (2019, background): “However, each dataset comes with specific biases (Torralba and Efros, 2011).”
Strike (With) a Pose: Neural Networks Are Easily Fooled by Strange Poses of Familiar Objects (2018, neutral): “The first is addressing biased data [49].”
Webly Supervised Joint Embedding for Cross-Modal Image-Text Retrieval (2018, background): “ scale well to the open-world setting as it is impracticable to collect and annotate images for all relevant concepts [33, 39]. Moreover, there exists different types of bias in the existing datasets [28, 49, 50]. In order to circumvent these issues, several recent studies focused on using web image”
Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings (2016, background): “Biases in the curation of machine learning data-sets have explored in [37, 4].”
Learning Visual Features from Large Weakly Supervised Data (2015, background): “…you re carrying a bag with some room it s nearly x x and weighs ounces new one is x x and weighs ounces i underexposed this one a bit did exposure bracketing script underexposure on that camera looks melty yummy gold kodak ﬁlm like not as impressive as embankment that s for sure ciﬁc task [37, 48].”
Predicting Deep Zero-Shot Convolutional Neural Networks Using Textual Descriptions (2015, background): “In [27], there is no target data available, however, the set of labels is still given and is consistent across domains.”