CLIP-CLOP: CLIP-Guided Collage and Photomontage
By Piotr Wojciech Mirowski, Dylan Banarse, Mateusz Malinowski, Simon Osindero, Chrisantha Fernando. 2022
18 citations recorded by the source corpus.
The unabated mystique of large-scale neural networks, such as the CLIP dual image-and-text encoder, popularized automatically generated art. Increasingly more sophisticated generators enhanced the artworks' realism and visual appearance, and creative prompt engineering enabled stylistic expression. Guided by an artist-in-the-loop ideal, we design a gradient-based generator to produce collages. It requires the human artist to curate libraries of image patches and to describe (with prompts) the whole image composition, with the option to manually adjust the patches' positions during generation, thereby allowing humans to reclaim some control of the process and achieve greater creative freedom. We explore the aesthetic potentials of high-resolution collages, and provide an open-source Google Colab as an artistic tool.
Subfields: Biophysics, Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition, Generative Models, Multimodal Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Graphics and Computer-Aided Design
In Computer Vision and Pattern Recognition
In Generative Models
In Multimodal Learning
Citations in the corpus
- CLIPDraw: Exploring Text-to-Drawing Synthesis through Language-Image Encoders (2021, methodology): “CLIP-CLOP (Mirowski et al., 2022) extends CLIPDraw to use image patches rather than strokes, and CLIPasso (Vinker et al.”