One Embedder, Any Task: Instruction-Finetuned Text Embeddings

By Hongjin Su, Weijia Shi, Jungo Kasai, Yizhong Wang, Yushi Hu, Mari Ostendorf, Wen-tau Yih, Wen-Tau Yih, Noah A. Smith, Luke S. Zettlemoyer and 1 more. Annual Meeting of the Association for Computational Linguistics, 2022

475 citations recorded by the source corpus.

We introduce INSTRUCTOR, a new method for computing text embeddings given task instructions: every text input is embedded together with instructions explaining the use case (e.g., task and domain descriptions). Unlike encoders from prior work that are more specialized, INSTRUCTOR is a single embedder that can generate text embeddings tailored to different downstream tasks and domains, without any further training. We first annotate instructions for 330 diverse tasks and train INSTRUCTOR on this multitask mixture with a contrastive loss. We evaluate INSTRUCTOR on 70 embedding evaluation tasks (66 of which are unseen during training), ranging from classification and information retrieval to semantic textual similarity and text generation evaluation. INSTRUCTOR, while having an order of magnitude fewer parameters than the previous best model, achieves state-of-the-art performance, with an average improvement of 3.4% compared to the previous best results on the 70 diverse datasets. Our analysis suggests that INSTRUCTOR is robust to changes in instructions, and that instruction finetuning mitigates the challenge of training a single model on diverse datasets. Our model, code, and data are available at https://instructor-embedding.github.io.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/63c0dbe2426f9c92f469151d1773e5265ae6580e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

C-Pack: Packaged Resources To Advance General Chinese Embedding (2023, methodology): “…must be optimized through pre-training (Liu and Shao, 2022; Wang et al., 2022a), negative sampling (Izacard et al., 2021; Wang et al., 2022a), and multi-task fine-tuning (Su et al., 2022; Asai et al., 2022; Sanh et al., 2021; Wei et al., 2021; Muennighoff et al., 2022b, 2023a; Chung et al., 2022).”
C-Pack: Packed Resources For General Chinese Embeddings (2023, methodology): “…it needs embedding-oriented pre-training to prepare the text encoder [16], contrastive learning with sophisticated negative sampling to improve the embedding’s discriminability [43], and instruction-based fine-tuning [7, 50] to integrate different representation capabilities of text embedding.”
Large Language Models for Information Retrieval: A Survey (2023, unknown): “This intriguing area remains open for further investigation and advancement.”
Towards General Text Embeddings with Multi-stage Contrastive Learning (2023, methodology): “For fine-tuning, we initially started with the three datasets used in E5 (Wang et al., 2022b) fine-tuning and gradually incorporated datasets from MEDI (Su et al., 2023) and BERRI (Asai et al., 2023) to investigate the potential benefits.”
REPLUG: Retrieval-Augmented Black-Box Language Models (2023, methodology): “Instead of relying only on existing neural dense retrieval models (Karpukhin et al., 2020a; Izacard et al., 2022a; Su et al., 2022), we further propose R E P LUG LSR (R E P LUG with LM-Supervised Retrieval), which adapts the retriever in R E P LUG by using the LM itself to provide supervision about…”
Task-aware Retrieval with Instructions (2022, methodology): “Concurrent to our work, Su et al. (2022) trains a single dual-encoder model trained on embedding tasks including retrieval tasks with instructions.”
MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries (2024, unknown): “5 (Xiao et al., 2023), jina-embeddings-v2-base-en (Günther et al., 2023), e5-base-v2 (Wang et al., 2022), and instructor-large (Su et al., 2023).”
Nomic Embed: Training a Reproducible Long Context Text Embedder (2024, methodology): “Recent systems such as E5 (Wang et al., 2022), GTE (Li et al., 2023), BGE (Xiao et al., 2023), Instruc-tOR (Su et al., 2023a), and Jina (G¨unther et al., 2023, 2024) utilize a multi-stage regime in which a pretrained transformer is first contrastively fine-tuned using a large corpus of weakly paired”
Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models (2025, unknown): “For instance, during the embedding model training process, incorporating differentiated tasks across aspects such as instruction type, domain, and language has yielded improved performance in downstream tasks (Su et al., 2023).”
LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders (2024, methodology): “…contrastive training followed by multi-task fine-tuning (Ni et al., 2022; Wang et al., 2022a; Li et al., 2023a; Xiao et al., 2023) Recent approaches have focused on enhancing the generalization and transferability of text embeddings using instructions (Su et al., 2023; Asai et al., 2023).”
Improving Text Embeddings with Large Language Models (2023, background): “For instance, Instructor [38] is only trained on instructions from 330 English datasets, whereas BGE [46] only focuses on high-resource languages such as English and Chinese.”
Nomic Embed Vision: Expanding the Latent Space (2024, methodology): “Since then, models such as E5 (Wang et al., 2024), GTE (Li et al., 2023), BGE (Xiao et al., 2024), InstructOR (Su et al., 2023), Jina (G¨unther et al., 2024), and Nomic (Nussbaum et al., 2024) train dual encoders in multiple stages.”
A Comparative Study on Affective Cues in Text Embeddings Across Psychological Emotion Theories (2026, unknown): “All encoders are multilingual, unless their training corpora are mainly in English ( † ). specifications included as additional input [47].”
IdioLink: Retrieving Meaning Beyond Words Across Idiomatic and Literal Expressions (2026, unknown): “More recent models scale this paradigm through larger architectures, multilingual training, and instruction-based supervision (Wang et al., 2024a,c; Su et al., 2023; Muennighoff et al., 2025).”
A Systematic Multi-Domain Evaluation of Document Retrievers (2026, unknown): “Retrievers. docT5query [58] expands documents by generating multiple queries for each and retrieving with BM25. query2doc [80] instead expands the query, generating a pseudo-document and again retrieving with BM25, while HyDE [19] generates a hypothetical document and retrieves with the Contriever.”