DALL-E: CREATING IMAGES FROM TEXT

By Mr. D Murahari, Reddy, Mr. Sk, Mr. M Masthan Basha, Mr. N Chinnaiahgari Hari, P. Student. 2021

Subfields: Generative Models, Multimodal Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Generative Models

In Multimodal Learning

Citations in the corpus

Symbolic Music Generation with Diffusion Models (2021, methodology): “Jukebox [11] and DALL-E [12] use similar approaches for text-conditioned generative music audio and image models.”
StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery (2021, background): “Recently, DALL·E [35, 36], a 12-billion parameter version of GPT-3 [3], which at 16-bit precision requires over 24GB of GPU memory, has shown a diverse set of capabilities in generating and applying transformations to images guided by text.”
Pretrained Transformers as Universal Computation Engines (2021, methodology): “Our work is most similar to DALL-E (Ramesh et al., 2021), which uses a single transformer to embed both the image and text modalities, which we consider to be generating a “universal latent space” that projects any type of input into a single latent space.”
Continual World: A Robotic Benchmark For Continual Reinforcement Learning (2021, background): “assumption, as for example in image processing [27, 46, 9, 38], language modelling [49, 10] or biological applications [44].”
Transformers in Vision: A Survey (2021, methodology): “The model is named DALL·E using a portmanteau of the Spanish artist Salvador Dalı́ and the Pixar’s blockbuster movie WALL·E. DALL·E model has 12 billion parameters and it is trained on a large set of text-image pairs taken from the internet.”
AudioGen: Textually Guided Audio Generation (2022, methodology): “DALL-E (Reddy et al., 2021) ﬁrst transforms the patches of an image to discrete codes using a pre-trained VQ-VAE.”
Text Data Augmentation for Deep Learning (2021, background): “Recent advances in generative modeling such as StyleGAN for images, GPT-3 for text [14], and DALL-E unifying both text and images [15], have been astounding.”
WenLan: Bridging Vision and Language by Large-Scale Multi-Modal Pre-Training (2021, background): “Particularly, in January 2021, OpenAI released a multi-modal version of GPT3 [3] called DALL·E [1], demonstrating its excellent textto-image generation capability.”
A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT (2023, methodology): “The Visualization Generator pattern allows the user to generate visualizations by producing textual outputs that can be fed to other tools, such as other AI-based image generators, like DALL-E [13].”
Fixing Data Augmentation to Improve Adversarial Robustness (2021, background): “Generative models, which are capable of creating novel images, are viable augmentation candidates (OpenAI, 2021).”
Continual Learning with Foundation Models: An Empirical Study of Latent Replay (2022, background): “A recent papers on natural language processing (NLP) indicate that SSL of large transformer models (Devlin et al., 2018; Reddy et al., 2021; Douillard et al., 2021) on vast amounts of pre-training data results in models that, without further training, generalize well to new tasks from few examples…”
Understanding and Creating Art with AI: Review and Outlook (2021, methodology): “In January 2021, OpenAI presented a 12-billion parameter network called DALL·E trained to generate images from text descriptions, using a large dataset of text-image pairs [1].”
Adversarial Text-to-Image Synthesis: A Review (2021, background): “…be to investigate and build upon progress made with other generative models such as Variational Autoencoders (VAEs) [147, 148], autoregressive models [106, 149, 150], ﬂow-based models [89, 90, 151], score matching networks [152, 153, 154, 155], and transformer-based models [156, 157, 158, 159].”