Modeling Visual Context is Key to Augmenting Object Detection Datasets

By Nikita Dvornik, Julien Mairal, Cordelia Schmid. 2018

279 citations recorded by the source corpus.

Performing data augmentation for learning deep neural networks is well known to be important for training visual recognition systems. By artificially increasing the number of training examples, it helps reducing overfitting and improves generalization. For object detection, classical approaches for data augmentation consist of generating images obtained by basic geometrical transformations and color changes of original training images. In this work, we go one step further and leverage segmentation annotations to increase the number of object instances present on training data. For this approach to be successful, we show that modeling appropriately the visual context surrounding objects is crucial to place them in the right environment. Otherwise, we show that the previous strategy actually hurts. With our context model, we achieve significant mean average precision improvements when few labeled examples are available on the VOC’12 benchmark.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

CutMix: Regularization Strategy to Train Strong Classifiers With Localizable Features (2019, methodology): “For object detection, object insertion methods [5, 4] have been proposed as a way to synthesize objects in the background.”
Not Using the Car to See the Sidewalk — Quantifying and Controlling the Effects of Context in Classification and Segmentation (2018, background): “Data augmentation by adding objects into new contexts was proposed in [8], to improve the performance of object detection models.”
Counterfactuals uncover the modular structure of deep generative models (2018, background): “Extrapolations moreover likely support adaptability to environmental changes and robust decision making (Dvornik et al., 2018).”
A Comprehensive Survey of Image Augmentation Techniques for Deep Learning (2022, background): “Instance-level CutPas [36], Scale and Blend [37], Context DA [38], Simple CutPas [39], Continuous CutPas [40].”
Learning Object Placement by Inpainting for Compositional Data Augmentation (2020, methodology): “The context-based cut-and-paste method most related to our work is [5], in that placement is learned based on context.”
Deep Learning for Generic Object Detection: A Survey (2018, unknown): “また,Dvornik ら [63] は,物体を適切な環境に配置するためには物体を囲む視覚コンテキストの適切なモデル化が重要であると示し,データ拡張用の適切な新物体配置位置を自動的に見つけるコンテキストモデルを提案した.”
Synthetic Data for Deep Learning (2019, background): “[158] argue that an important problem for this type of data augmentation is to preserve visual context, i.”
Improving Augmentation and Evaluation Schemes for Semantic Image Synthesis (2020, methodology): “Inspired by the task-specific augmentation studies in other vision applications [15, 14, 51, 3, 59], in this work, we propose an augmentation method specifically designed to overcome the above mentioned limitations of the SIS models.”
Imbalance Problems in Object Detection: A Review (2019, background): “Alternatively, artificial images can be produced to augment the dataset [64], [65], [99], [100] by generating composite images in which multiple crops and/or images are blended.”