Dataset Condensation with Differentiable Siamese Augmentation
By Bo Zhao, Hakan Bilen. International Conference on Machine Learning, 2021
410 citations recorded by the source corpus.
In many machine learning problems, large-scale datasets have become the de-facto standard to train state-of-the-art deep networks at the price of heavy computation load. In this paper, we focus on condensing large training sets into significantly smaller synthetic sets which can be used to train deep neural networks from scratch with minimum drop in performance. Inspired from the recent training set synthesis methods, we propose Differentiable Siamese Augmentation that enables effective use of data augmentation to synthesize more informative synthetic images and thus achieves better performance when training networks with augmentations. Experiments on multiple image classification benchmarks demonstrate that the proposed method obtains substantial gains over the state-of-the-art, 7% improvements on CIFAR10 and CIFAR100 datasets. We show with only less than 1% data that our method achieves 99.6%, 94.9%, 88.5%, 71.5% relative performance on MNIST, FashionMNIST, SVHN, CIFAR10 respectively. We also explore the use of our method in continual learning and neural architecture search, and show promising results.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition
View Paper: https://www.semanticscholar.org/paper/99e93357b8b3ffd309d2dbae187f61143b928abf
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Hakan Bilen
- Bo Zhao
- Timothy Nguyen
- Jaehoon Lee
- Lechao Xiao
- Roman Novak
- Greg Yang
- Jascha Narain Sohl-Dickstein
In Computer Vision and Pattern Recognition
- Bo Zhao
- Hakan Bilen
- Jaehoon Lee
- Timothy Nguyen
- Roman Novak
- Lechao Xiao
- Antonio Torralba
- Alexei A. Efros
Citations in the corpus
- Dataset Condensation with Distribution Matching (2021, methodology): “We also compare to four state-of-the-art training set synthesis methods, namely, DD [45], LD [6], DC [50] and DSA [49].”
- Dataset Distillation with Infinitely Wide Convolutional Networks (2021, methodology): “6% by Zhao and Bilen [2021]. Finally, in Figure 4, we investigate the performance of KIP images over the course of training of a single run, as compared to natural images, over a range of hyperparameters. We find the outperformance of KIP images above natural images consistent across hyperparameters”