Jina CLIP: Your CLIP Model Is Also Your Text Retriever

By Andreas Koukounas, Georgios Mastrapas, Michael Günther, Bo Wang, Scott Martens, Isabelle Mohr, Saba Sturua, Mohammad Kalim Akram, Joan Fontanals Mart'inez, Saahil Ognawala and 4 more. 2024

Subfields: Information Retrieval & Search, Multimodal Learning, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Retrieval & Search

1. Andriy Mulyar
2. Zach Nussbaum
3. Brandon Duderstadt

In Multimodal Learning

1. Andriy Mulyar
2. Zach Nussbaum
3. Brandon Duderstadt

In Representation & Self-Supervised Learning

1. Andriy Mulyar
2. Zach Nussbaum
3. Brandon Duderstadt

Citations in the corpus

• Nomic Embed Vision: Expanding the Latent Space (2024, background): “Recently, Jina CLIP v1 (Koukounas et al., 2024) was introduced to address this issue.”
• BRIDGE: Multimodal-to-Text Retrieval via Reinforcement-Learned Query Alignment (2026, unknown): “• Jina-CLIP [18]: Multi-task contrastive model supporting both text-image and text-text retrieval.”
• MARVEL: Multimodal Adaptive Reasoning-intensiVe Expand-rerank and retrievaL (2026, unknown): “• Jina-CLIP [18]: Multi-task contrastive model supporting both text-image and text-text retrieval.”
• Mieb: Massive Image Embedding Benchmark (2025, unknown): “We evaluate many models across this line of research including CLIP, SigLIP [68], ALIGN [25], Jina-CLIP [30], DataComp-CLIP [18], Open-CLIP [10], and Eva-CLIP [50].”
• Nemotron ColEmbed V2: Top-Performing Late Interaction embedding models for Visual Document Retrieval (2026, unknown): “VLM decoder models have been adapted as contrastive embedding models for visual document retrieval, like Jina CLIP [17] and Nomic Embed Vision [18].”
• Document Haystacks: Vision-Language Reasoning Over Piles of 1000+ Documents (2024, unknown): “For computing the text-to-image similarities, we employed the Jina-CLIP-v1 [18] variant, Nomic-Embed-Vision-v1.”
• jina-embeddings-v4: Universal Embeddings for Multimodal Multilingual Retrieval (2025, unknown): “The semantic embedding paradigm can also encompass models that support more than one modality, like bimodal image-text models, including OpenAI’s CLIP [Rad-ford et al., 2021] and subsequent developments including jina-clip [Koukounas et al., 2024].”