Benchmark for Compositional Text-to-Image Synthesis

By Dong Huk Park, Samaneh Azadi, Xihui Liu, Trevor Darrell, Anna Rohrbach. 2021

165 citations recorded by the source corpus.

Subfields: Multimodal Learning, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Multimodal Learning

In Evaluation & Benchmarks

Citations in the corpus

T2I-CompBench: A Comprehensive Benchmark for Open-world Compositional Text-to-image Generation (2023, unknown): “Previous work focused on concept conjunction and negation [7], attribute binding with colors [8, 9, 34], and spatial relationships between objects [35, 10].”
Zero-Shot Text-Guided Object Generation with Dream Fields (2021, methodology): “Instead, we use the CLIP R-Precision metric [41] from the text-to-image generation literature to measure how well rendered images align with the true caption.”
TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answering (2023, methodology): “CLIPScore [17] and CLIP-R [39] are based on the cosine similarity of image and text CLIP [42] embeddings.”
Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesis (2022, background): “(Park et al., 2021).”
CLIP-Mesh: Generating textured meshes from text using pretrained image-text models (2022, methodology): “During evaluation they are rendered from a held-out pose not seen during training, a CLIP-R precision score [Park et al. 2021] is then computed between the held-out pose renderings and the captions used to generate the shapes.”
Point-E: A System for Generating 3D Point Clouds from Complex Prompts (2022, methodology): “Here we use the ViT-L/14 CLIP model to compute CLIP R-Precision, but we report results with an alternative CLIP model in Section 5.3.”
Follow Your Pose: Pose-Guided Text-to-Video Generation using Pose-Free Videos (2023, methodology): “1) CLIP score : We follow [10] to evaluate our approach on CLIP score [21, 22] or video-text alignment.”
Imagen Editor and EditBench: Advancing and Evaluating Text-Guided Image Inpainting (2022, unknown): “CLIP-R-Prec is a ranking based approach (typically formulated as text RPrecision [28]) that measures how well the generated image retrieves the text prompt with CLIP from among a set of text distractors.”
Unleashing the Power of Edge-Cloud Generative AI in Mobile Networks: A Survey of AIGC Services (2023, unknown): “2) Frechet Inception Distance: The Frechet Inception Distance (FID) has emerged as a well-established metric for evaluating the effectiveness of generative models, particularly GANs, in terms of image quality and diversity [56].”
GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment (2023, background): “Many of these are reference-based: BLEU [28] and CIDEr [40] can be used to evaluate generations by performing captioning and comparing to a set of reference captions [17, 18], while R-Precision [29, 43] measures text recall scores from a reference dataset of captions.”
Pix2Video: Video Editing using Image Diffusion (2023, methodology): “To capture the faithfulness, we follow [18, 57] and report CLIP score [17, 34], which is the cosine similarity between the CLIP embedding [37] of the edit prompt and the embedding of each frame in the edited video.”
Shap-E: Generating Conditional 3D Implicit Functions (2023, methodology): “As done by Jain et al. [26] and various follow-up literature, we compute CLIP R-Precision [42] on a set of COCO validation prompts.”
DF-GAN: A Simple and Effective Baseline for Text-to-Image Synthesis (2020, background): “Second, existing studies usually ﬁx the extra networks [33,50] during the adversarial training, making these networks easily fooled by the generator to synthesize adversarial features [30, 52], thereby weakening their supervision power on semantic consistency.”
Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation (2022, unknown): “1) CLIP score: We follow [14] to evaluate Tune-A-Video on CLIP score [13, 27] for videotext alignment.”
Imagen Video: High Definition Video Generation with Diffusion Models (2022, methodology): “…our model development process, we evaluated Imagen Video on several different metrics, such as FID on individual frames (Heusel et al., 2017), FVD (Unterthiner et al., 2019) for temporal consistency, and frame-wise CLIP scores (Hessel et al., 2021; Park et al., 2021) for video-text alignment.”