Multimodal Learning
Vision-language models, CLIP, text-to-image, cross-modal representation.
A topic in the Science corpus, drawn from 1,221 papers.
Trusted voices in Multimodal Learning
Ordered by the trust Multimodal Learning's own network concentrates on each researcher, most first.
- Li Fei-Fei — trusted by 1,083 researchers here
- Andrej Karpathy — trusted by 897 researchers here
- Devi Parikh — trusted by 792 researchers here
- Dhruv Batra — trusted by 695 researchers here
- Ryan Kiros — trusted by 584 researchers here
- Richard S. Zemel — trusted by 545 researchers here
- Jiasen Lu — trusted by 479 researchers here
- Ruslan Salakhutdinov — trusted by 470 researchers here
- Mohit Bansal — trusted by 437 researchers here
- Tamara L. Berg — trusted by 354 researchers here
- Marcus Rohrbach — trusted by 365 researchers here
- Hao Hao Tan — trusted by 370 researchers here
Most-cited papers in Multimodal Learning
Ordered by the recorded citation count from the source corpus; a citation is not an endorsement.
- Learning Transferable Visual Models From Natural Language Supervision (2021, International Conference on Machine Learning) — 52,559 citations
- The Llama 3 Herd of Models (2024) — 17,085 citations
- Show, Attend and Tell: Neural Image Caption Generation with Visual Attention (2015) — 10,867 citations
- Visual Instruction Tuning (2023, Neural Information Processing Systems) — 10,396 citations
- Hierarchical Text-Conditional Image Generation with CLIP Latents (2022, arXiv.org) — 9,210 citations
- BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models (2023) — 8,665 citations
- Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding (2022) — 8,628 citations
- Adding Conditional Control to Text-to-Image Diffusion Models (2023) — 7,354 citations
- BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation (2022) — 7,032 citations
- Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations (2016) — 6,638 citations
- Show and tell: A neural image caption generator (2014) — 6,597 citations
- Zero-Shot Text-to-Image Generation (2021) — 6,590 citations
- VQA: Visual Question Answering (2015) — 6,567 citations
- Flamingo: a Visual Language Model for Few-Shot Learning (2022, Neural Information Processing Systems) — 6,220 citations
- Deep Visual-Semantic Alignments for Generating Image Descriptions (2014) — 6,086 citations
- Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision (2021) — 5,764 citations
- Improved Baselines with Visual Instruction Tuning (2023) — 5,499 citations
- LAION-5B: An open large-scale dataset for training next generation image-text models (2022) — 5,389 citations
- ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks (2019) — 4,585 citations
- Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection (2023) — 4,547 citations
- Multimodal Machine Learning: A Survey and Taxonomy (2017) — 4,410 citations
- DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation (2022) — 4,398 citations
- Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering (2016) — 4,360 citations
- FiLM: Visual Reasoning with a General Conditioning Layer (2017) — 4,130 citations
- Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context (2024, arXiv.org) — 3,818 citations
- DreamFusion: Text-to-3D using 2D Diffusion (2022) — 3,724 citations
- InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning (2023) — 3,705 citations
- Multimodal Deep Learning (2011) — 3,550 citations
- Sigmoid Loss for Language Image Pre-Training (2023) — 3,447 citations
- Generative Adversarial Text to Image Synthesis (2016) — 3,433 citations
- MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models (2023) — 3,250 citations
- DeViSE: A Deep Visual-Semantic Embedding Model (2013) — 3,053 citations
- Conceptual Captions: A Cleaned, Hypernymed, Image Alt-text Dataset For Automatic Image Captioning (2018) — 3,041 citations
- CLIPScore: A Reference-free Evaluation Metric for Image Captioning (2021) — 3,009 citations
- LXMERT: Learning Cross-Modality Encoder Representations from Transformers (2019) — 2,971 citations
- PaLM-E: An Embodied Multimodal Language Model (2023) — 2,915 citations
- Prompt-to-Prompt Image Editing with Cross Attention Control (2022) — 2,848 citations
- Align before Fuse: Vision and Language Representation Learning with Momentum Distillation (2021) — 2,847 citations
- An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion (2022) — 2,832 citations
- Aligning Books and Movies: Towards Story-Like Visual Explanations by Watching Movies and Reading Books (2015) — 2,762 citations
- UNITER: UNiversal Image-TExt Representation Learning (2019) — 2,659 citations
- Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models (2015) — 2,641 citations
- MSR-VTT: A Large Video Description Dataset for Bridging Video and Language (2016) — 2,526 citations
- Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering (2022) — 2,503 citations
- MMMU: A Massive Multi-Discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI (2023) — 2,391 citations
- VisualBERT: A Simple and Performant Baseline for Vision and Language (2019) — 2,355 citations
- ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision (2021) — 2,347 citations
- SPICE: Semantic Propositional Image Caption Evaluation (2016) — 2,344 citations
- Conditional Prompt Learning for Vision-Language Models (2022) — 2,313 citations
- MMBench: Is Your Multi-modal Model an All-around Player? (2023) — 2,293 citations
Most active researchers
By papers in this topic: a volume measure, not trust.
- Jianfeng Gao — 33 papers
- Zhe Gan — 32 papers
- Devi Parikh — 30 papers
- Lijuan Wang — 29 papers
- Linjie Li — 24 papers
- Dhruv Batra — 23 papers
- Marcus Rohrbach — 21 papers
- Antonio Torralba — 20 papers
- Mohit Bansal — 20 papers
- Zicheng Liu — 20 papers
- Qi Wu — 19 papers
- Jianwei Yang — 18 papers