LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop

By Fisher Yu, Yinda Zhang, Shuran Song, Ari Seff, Jianxiong Xiao. 2015

2,676 citations recorded by the source corpus.

While there has been remarkable progress in the performance of visual recognition algorithms, the state-of-the-art models tend to be exceptionally data-hungry. Large labeled training datasets, expensive and tedious to produce, are required to optimize millions of parameters in deep network models. Lagging behind the growth in model capacity, the available datasets are quickly becoming outdated in terms of size and density. To circumvent this bottleneck, we propose to amplify human effort through a partially automated labeling scheme, leveraging deep learning with humans in the loop. Starting from a large set of candidate images for each category, we iteratively sample a subset, ask people to label them, classify the others with a trained model, split the set into positives, negatives, and unlabeled based on the classification confidence, and then iterate with the unlabeled set. To assess the effectiveness of this cascading procedure and enable further progress in visual recognition research, we construct a new image dataset, LSUN. It contains around one million labeled images for each of 10 scene categories and 20 object categories. We experiment with training popular convolutional networks and find that they achieve substantial performance gains when trained on this dataset.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Evaluation & Benchmarks

Citations in the corpus

Denoising Diffusion Probabilistic Models (2020, background): “4 for LSUN 256 × 256 samples [63], and Appendix C for more.”
CutMix: Regularization Strategy to Train Strong Classifiers With Localizable Features (2019, methodology): “In Table 12, we report the averaged OOD detection performances against seven out-of-distribution samples from [13, 21], including TinyImageNet, LSUN [46], uniform noise, Gaussian noise, etc.”
Analyzing and Improving the Image Quality of StyleGAN (2019, neutral): “99 , (cid:15) = 10 − 8 , minibatch = 32 ), and training datasets [24, 44].”
A Style-Based Generator Architecture for Generative Adversarial Networks (2018, background): “Figures 10, 11, and 12 show an uncurated set of re-sults for LSUN [62] B EDROOM , C ARS , and C ATS , respectively.”
Glow: Generative Flow with Invertible 1x1 Convolutions (2018, methodology): “For Imagenet, we use the 32 × 32 and 64 × 64 downsampled version of ImageNet (Oord et al., 2016), and for LSUN we downsample to 96 × 96 and take random crops of 64 × 64 .”
Progressive Growing of GANs for Improved Quality, Stability, and Variation (2017, methodology): “Since 100k images is a very limited amount of training data for most categories, we enabled mirror augmentation in these tests (but not for BEDROOM or DOG ).”
Projected GANs Converge Faster (2021, methodology): “We show the relative FD at different layers of F ( rel - FD i ) between 50k generated and real images on LSUN-Church. rel - FD i is normalized using the baseline Fréchet Distances for a model with a standard single RGB image discriminator.”
On Fast Sampling of Diffusion Probabilistic Models (2021, methodology): “, 2015)), and LSUN-bedroom (∼3M bedroom images of resolution 256× 256 (Yu et al., 2015)).”
Cascaded Diffusion Models for High Fidelity Image Generation (2021, background): “Finally, we verify that conditioning augmentation is also effective on the LSUN dataset (Yu et al., 2015) and therefore is not specific to ImageNet (Section 4.”
Diffusion Models Beat GANs on Image Synthesis (2021, methodology): “To evaluate our improved model architecture on unconditional image generation, we train separate diffusion models on three LSUN [71] classes: bedroom, horse, and cat.”
On Buggy Resizing Libraries and Surprising Subtleties in FID Calculation (2021, methodology): “We plot FID as a function of JPEG compression, applied to StyleGAN2 images [38], trained on LSUN Churches [76] (left) and FFHQ [37] (right) at a resolution of 256 × 256.”
Noise Estimation for Generative Diffusion Models (2021, methodology): “We trained our model Pθ on three image datasets (i) CelebA 64x64 [20], (ii) LSUN Bedroom 256x256, and (iii) LSUN Church 256x256 [36].”
Taming Transformers for High-Resolution Image Synthesis (2020, methodology): “1 reports results for unconditional image modeling on ImageNet (IN) [14], Restricted ImageNet (RIN) [57], consisting of a subset of animal classes from ImageNet, LSUN Churches and Towers (LSUN-CT) [69], and for conditional image modeling of RIN conditioned on depth maps obtained with the approach of”
Score-Based Generative Modeling through Stochastic Differential Equations (2020, neutral): “256ˆ 256 LSUN bedroom and church outdoor (Yu et al., 2015) datasets with the VE SDE and our continuous objective Eq.”