Task-aware Retrieval with Instructions

By Akari Asai, Timo Schick, Patrick Lewis, Xilun Chen, Gautier Izacard, Sebastian Riedel, Hannaneh Hajishirzi, Wen-tau Yih. 2022

139 citations recorded by the source corpus.

We study the problem of retrieval with instructions, where users of a retrieval system explicitly describe their intent along with their queries. We aim to develop a general-purpose task-aware retrieval system using multi-task instruction tuning, which can follow human-written instructions to find the best documents for a given query. We introduce the first large-scale collection of approximately 40 retrieval datasets with instructions, BERRI, and present TART, a multi-task retrieval system trained on BERRI with instructions. TART shows strong capabilities to adapt to a new retrieval task via instructions and advances the state of the art on two zero-shot retrieval benchmarks, BEIR and LOTTE, outperforming models up to three times larger. We further introduce a new evaluation setup, X^2-Retrieval to better reflect real-world scenarios, where diverse domains and tasks are pooled and a system needs to find documents aligning users' intents. In this setup, TART significantly outperforms competitive baselines, further demonstrating the effectiveness of guiding retrieval with instructions.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Information Retrieval & Search

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Information Retrieval & Search

Citations in the corpus

Precise Zero-Shot Dense Retrieval without Relevance Labels (2022, unknown): “Concurrent to us, Asai et al. (2022) studied “Task-aware Retrieval with Instructions”.”
LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders (2024, background): “…contrastive training followed by multi-task fine-tuning (Ni et al., 2022; Wang et al., 2022a; Li et al., 2023a; Xiao et al., 2023) Recent approaches have focused on enhancing the generalization and transferability of text embeddings using instructions (Su et al., 2023; Asai et al., 2023).”
Retrieval-based Language Models and Applications (2023, methodology): “, 2022a) • Task-aware Retrieval with Instructions (Asai et al., 2022)”
NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models (2024, unknown): “It has also been recently applied for training retrievers and general-purpose embedding models that can adapt their output embeddings with different instructions and task types (Asai et al., 2022; Wang et al., 2023b).”
MIMIC-IT: Multi-Modal In-Context Instruction Tuning (2023, background): “The recent advancements in artificial intelligence have focused on conversational assistants [42, 31, 30, 13, 17] that possess a strong ability to understand user intentions [35] and then execute actions [5, 51].”
Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting (2023, methodology): “For supervised methods, in addition to the base-lines in Section 4.2, we add TART (Asai et al., 2023), a supervised instruction-tuned passage re-ranker trained on 37 datasets, including over 5 million instances.”
One Embedder, Any Task: Instruction-Finetuned Text Embeddings (2022, background): “Concurrent work demonstrated that instructions could facilitate information retrieval (Asai et al., 2022), which is related to our I NSTRUCT OR design.”
Large Language Models for Information Retrieval: A Survey (2023, unknown): “Inspired by the LLM’s excellent capability to encode and comprehend natural language, some researchers [130–132] leverage LLMs as retrieval encoders and investigate the impact of model scale on retriever performance.”
Towards General Text Embeddings with Multi-stage Contrastive Learning (2023, methodology): “We totally used ∼ 3M pairs for fine-tuning, which is a combination of training data used by previous research (Gao et al., 2021; Gao and Callan, 2022; Asai et al., 2023; Su et al., 2023; Li et al., 2023).”
Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy (2023, methodology): “We used TART (Asai et al., 2022) as the re-ranker, and distilled knowledge from TART to the dense retriever for no more than 1,000 steps.”
C-Pack: Packaged Resources To Advance General Chinese Embedding (2023, methodology): “, 2020), and instruction-based fine-tuning (Su et al., 2022; Asai et al., 2022) to integrate different representation capabilities of text embedding.”
Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection (2023, background): “Recent or concurrent work studies instruction-tuning of retrieval systems (Asai et al., 2023b) or joint training of retrieval and LM components (Lin et al., 2023), while we leave exploring the effectivess of such appraoches for future work.”
Recommendation as Instruction Following: A Large Language Model Empowered Recommendation Approach (2023, unknown): “Despite the similar efforts in prior studies [2, 13], we discuss the expression of user needs in the”
Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agent (2023, methodology): “• TART (Asai et al., 2022): A supervised instruction-tuned passage re-ranker trained on 37 datasets, including over 5 million instances.”
IdioLink: Retrieving Meaning Beyond Words Across Idiomatic and Literal Expressions (2026, unknown): “We assess encoder models: Contriever-based TART (Asai et al., 2023), the instruction-tuned multilingual E5 (Wang et al., 2024c), and Nomic-Embed-v2 (Nussbaum and Duderstadt, 2025).”
When Is 0.1% Enough? Analyzing the Combined Effects of Dimensionality Reduction and Quantization on Text Embedding Compression (2026, unknown): “…models have been developed using large-scale contrastive learning, weak supervision, task-specific prefixes, or natural language instructions (Wang et al., 2022; Su et al., 2023; Asai et al., 2023; Li et al., 2023; Xiao et al., 2024; Nussbaum et al., 2025; Wang et al., 2024; Lee et al., 2025a).”
DRAMA: Diverse Augmentation from Large Language Models to Smaller Dense Retrievers (2025, unknown): “Many recent models additionally optimize for broader text embedding tasks such as clustering and classification as well as instruction following or reasoning-intensive retrieval (Lee et al., 2024; Wang et al., 2024c; Su et al., 2023; Asai et al., 2023; Weller et al., 2025; Shao et al., 2025).”