Co-Mixup: Saliency Guided Joint Mixup with Supermodular Diversity

By Jang-Hyun Kim, Wonho Choo, Hosan Jeong, Hyun Oh Song. 2021

215 citations recorded by the source corpus.

While deep neural networks show great performance on fitting to the training distribution, improving the networks' generalization performance to the test distribution and robustness to the sensitivity to input perturbations still remain as a challenge. Although a number of mixup based augmentation strategies have been proposed to partially address them, it remains unclear as to how to best utilize the supervisory signal within each input data for mixup from the optimization perspective. We propose a new perspective on batch mixup and formulate the optimal construction of a batch of mixup data maximizing the data saliency measure of each individual mixup data and encouraging the supermodular diversity among the constructed mixup data. This leads to a novel discrete optimization problem minimizing the difference between submodular functions. We also propose an efficient modular approximation based iterative submodular minimization algorithm for efficient mixup computation per each minibatch suitable for minibatch based neural network training. Our experiments show the proposed method achieves the state of the art generalization, calibration, and weakly supervised localization results compared to other mixup methods. The source code is available at https://github.com/snu-mllab/Co-Mixup.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Sungmin Cha
2. Taesup Moon
3. Beomyoung Kim
4. Sook Yoon
5. Alvaro Fuentes
6. Mingle Xu
7. Dong Sun Park
8. Y. Yoo

In Computer Vision and Pattern Recognition

1. Sungmin Cha
2. Taesup Moon
3. Beomyoung Kim
4. Sook Yoon
5. Alvaro Fuentes
6. Mingle Xu
7. Dong Sun Park
8. Y. Yoo

Citations in the corpus

• SSUL: Semantic Segmentation with Unknown Label for Exemplar-based Class-Incremental Learning (2021, background): “For example, the weakly-supervised semantic segmentation approaches [29, 31, 14] generate pseudo-labels filtering out the background regions using saliency map, and recent data augmentations [17, 16] also exploit the saliency information to find the optimal mixing of mask.”
• A Comprehensive Survey of Image Augmentation Techniques for Deep Learning (2022, background): “Non-instance-level SamplePairing [27], Mixup [28], BC Learning [29], CutMix [30], Mo-saic [22], AugMix [31], PuzzleMix [32], Co-Mixup [33], SuperMix [34], GridMix [35].”