LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs

By Christoph Schuhmann, R. Vencu, Romain Beaumont, R. Kaczmarczyk, Clayton Mullis, Aarush Katta, Theo Coombes, Jenia Jitsev, Aran Komatsuzaki. 2021

1,887 citations recorded by the source corpus.

Multi-modal language-vision models trained on hundreds of millions of image-text pairs (e.g. CLIP, DALL-E) gained a recent surge, showing remarkable capability to perform zero- or few-shot learning and transfer even in absence of per-sample labels on target image data. Despite this trend, to date there has been no publicly available datasets of sufficient scale for training such models from scratch. To address this issue, in a community effort we build and release for public LAION-400M, a dataset with CLIP-filtered 400 million image-text pairs, their CLIP embeddings and kNN indices that allow efficient similarity search.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Multimodal Learning, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Multimodal Learning

In Evaluation & Benchmarks

Citations in the corpus

High-Resolution Image Synthesis with Latent Diffusion Models (2021, methodology): “45B parameter model conditioned on language prompts on LAION-400M [76].”
Flamingo: a Visual Language Model for Few-Shot Learning (2022, methodology): “We ablate replacing our image-text pairs (ITP) by the publicly available LAION-400M dataset [96], which leads to a slight degradation in performance.”
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation (2022, methodology): “1We only download images whose shorter edge is larger than 256 pixels from the original LAION400M.”
MAGMA - Multimodal Augmentation of Generative Models through Adapter-based Finetuning (2021, methodology): “…down-stream generalization capabilities [6, 15, 37, 60], prompted us to construct a large-scale pretraining dataset from various publicly available image-text datasets, including a ﬁltered version of LAION [43], Wikipedia Image-Text [48], CC3M [8], Visual Genome [26] and Localized Narratives [35].”
Scaling Up Vision-Language Pretraining for Image Captioning (2021, methodology): “Besides CC12M, there also exist some other largescale image-text datasets, such as WIT [36], WenLan [16], LAION-400M [34], and the datasets used in CLIP [33] and ALIGN [17].”
LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention (2023, background): “PT denotes pre-training on additional datasets [8, 29, 58, 7, 57], FT denotes fine-tuning on COCO Caption.”
Language Is Not All You Need: Aligning Perception with Language Models (2023, methodology): “Image-Caption Pairs The image-caption pairs are constructed from several datasets, including English LAION-2B [SBV22], LAION-400M [SVB21], COYO-700M [BPK22], and Conceptual Captions [SDGS18, CSDS21].”
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models (2023, positive): “…use the same pre-training dataset as BLIP with 129M images in total, including COCO (Lin et al., 2014), Visual Genome (Krishna et al., 2017), CC3M (Sharma et al., 2018), CC12M (Changpinyo et al., 2021), SBU (Ordonez et al., 2011), and 115M images from the LAION400M dataset (Schuhmann et al., 2021).”
Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding (2022, methodology): “However, a recent audit of another one of our data sources, LAION-400M [64], uncovered a wide range of inappropriate content including pornographic imagery, racist slurs, and harmful social stereotypes [4].”
The ROOTS Search Tool: Data Transparency for LLMs (2023, background): “In the Computer Vision community, related efforts 5 target large text and image datasets such as LAION (Schuhmann et al., 2022, 2021).”
Measuring Data (2022, background): “Analysing multimodal datasets, such as the recent large dataset LAION (Schuhmann et al., 2021), presents unique measurement challenges that we also do not address: Different modalities of a dataset can interact to result in novel harms and biases, as illustrated by Birhane et al. (2021).”
Data Governance in the Age of Large-Scale Data-Driven Language Technology (2022, background): “[125] Christoph Schuhmann, Richard Vencu, Romain Beaumont, Robert Kaczmarczyk, Clayton Mullis, Aarush Katta, Theo Coombes, Jenia Jitsev, and Aran Komatsuzaki.”
Infinity∞: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesis (2024, neutral): “[54], COYO [9], OpenImages [33].”
RSGPT: A Remote Sensing Vision Language Model and Benchmark (2023, neutral): “datasets, including LAION400M [49] and CC3 [50], and utilizing the BLIP2 model [23] for automatic image caption generation.”
Towards Universal Fake Image Detectors that Generalize Across Generative Models (2023, methodology): “The real class consists of images from LAION dataset [47].”