REPAIR: Removing Representation Bias by Dataset Resampling

By Nuno Vasconcelos. 2019

311 citations recorded by the source corpus.

Modern machine learning datasets can have biases for certain representations that are leveraged by algorithms to achieve high performance without learning to solve the underlying task. This problem is referred to as “representation bias”. The question of how to reduce the representation biases of a dataset is investigated and a new dataset REPresentAtion bIas Removal (REPAIR) procedure is proposed. This formulates bias minimization as an optimization problem, seeking a weight distribution that penalizes examples easy for a classifier built on a given feature representation. Bias reduction is then equated to maximizing the ratio between the classification loss on the reweighted dataset and the uncertainty of the ground-truth class labels. This is a minimax problem that REPAIR solves by alternatingly updating classifier parameters and dataset resampling weights, using stochastic gradient descent. An experimental set-up is also introduced to measure the bias of any dataset for a given representation, and the impact of this bias on the performance of recognition models. Experiments with synthetic and action recognition data show that dataset REPAIR can significantly reduce representation bias, and lead to improved generalization of models trained on REPAIRed datasets. The tools used for characterizing representation bias, and the proposed dataset REPAIR algorithm, are available at https://github.com/JerryYLi/Dataset-REPAIR/.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Interpretations are useful: penalizing explanations to align neural networks with prior knowledge (2019, neutral): “Similar to a previous study (Li & Vasconcelos, 2019), we transform the MNIST dataset to include three color channels and assign each class a distinct color, as shown in Fig.”
Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics (2020, background): “While AFLite, among others (Li and Vasconcelos, 2019; Gururangan et al., 2018), advocate removing “easy” instances from the dataset, our work shows that easy-to-learn instances can be useful.”
Unbiased Scene Graph Generation From Biased Training (2020, neutral): “augmentation or re-sampling [9, 25, 27, 11, 3], 2) unbiased learning through elaborately designed training curriculums or learning losses [70, 30], 3) disentangling biased representations from the unbiased [35, 4].”
Bias in machine learning - what is it good for? (2020, methodology): “Another approach to address biased models is to debias the data used to train the model, for example by removing biased parts, such as suggested for word embeddings [6], by oversampling [17], or by resampling [30].”
AGQA: A Benchmark for Compositional Spatio-Temporal Reasoning (2021, background): “Due to biases in answer distributions and the non-uniform distribution of occurrences of visual events, models may develop “cheating” approaches that can superficially guess answers without learning the underlying compositional reasoning process [36, 54].”
Contextualizing Multiple Tasks via Learning to Decompose (2021, methodology): “Similar to [3, 16, 23], which plot RGB colors over the grayscale images based on the Kuzushiji-49 dataset [9] and test whether a model will be disturbed by the colors.”
Towards causal benchmarking of bias in face analysis algorithms (2020, background): “protected by the law [44]. Approaches to mitigating dataset bias include collecting more thorough examples [54], using image synthesis to compensate for distribution gaps [46], and example resampling [48]. The machine learning community is active in analyzing biases of learning models, and how one m”
DataComp: In search of the next generation of multimodal datasets (2023, background): “Previous literature such as work by Le Bras et al. [81], Li & Vasconcelos [84], Gururangan et al. [50] advocate for removing easy instances from the training data.”
Adversarial Filters of Dataset Biases (2020, methodology): “Li & Vasconcelos (2019) recently proposed REPAIR, a method to remove representation bias by dataset resampling.”
Measuring Bias (2023, background): “Therefore, data bias is defined as a systematic distortion in the dataset samples that impairs its representativity [21], [27].”
Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigation (2019, background): “Classical literature on mitigating bias generally operates on simpler (often linear) models [11, 50, 28], which are easier to understand and control; only recently have researchers begun looking at mitigating bias in end-to-end trained deep learning models [16, 2, 40, 18, 48, 25, 30, 36, 47, 17].”
Learning De-biased Representations with Biased Representations (2019, methodology): “Existing methods attempt to remove a model’s dependency on bias by de-biasing the training data through data augmentation (Geirhos et al., 2019) or re-sampling tactics (Li & Vasconcelos, 2019).”