MTEB: Massive Text Embedding Benchmark

By Niklas Muennighoff, Nouamane Tazi, L. Magne, Nils Reimers. 2022

963 citations recorded by the source corpus.

Text embeddings are commonly evaluated on a small set of datasets from a single task not covering their possible applications to other tasks. It is unclear whether state-of-the-art embeddings on semantic textual similarity (STS) can be equally well applied to other tasks like clustering or reranking. This makes progress in the field difficult to track, as various models are constantly being proposed without proper evaluation. To solve this problem, we introduce the Massive Text Embedding Benchmark (MTEB). MTEB spans 8 embedding tasks covering a total of 58 datasets and 112 languages. Through the benchmarking of 33 models on MTEB, we establish the most comprehensive benchmark of text embeddings todate. We find that no particular text embedding method dominates across all tasks. This suggests that the field has yet to converge on a universal text embedding method and scale it up sufficiently to provide state-of-theart results on all embedding tasks. MTEB comes with open-source code and a public leaderboard at https://github.com/embeddings-benchmark/mteb.

Subfields: Artificial Intelligence, Representation & Self-Supervised Learning, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Representation & Self-Supervised Learning

In Evaluation & Benchmarks

Citations in the corpus

BLOOM: A 176B-Parameter Open-Access Multilingual Language Model (2022, methodology): “Additional analysis is provided in Muennighoff et al. (2022b).”
One Embedder, Any Task: Instruction-Finetuned Text Embeddings (2022, methodology): “5 All of these baselines are based on pretrained language models, achieving strong performance on the MTEB leader-board.”
C-Pack: Packaged Resources To Advance General Chinese Embedding (2023, methodology): “The classification task reuses the logistic regression classifier from MTEB (Muennighoff et al., 2022a), where the provided label is predicted based on the input embedding.”
NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models (2024, neutral): “We evaluate the proposed NV-Embed model on the full MTEB benchmark (Muennighoff et al., 2022) across 56 tasks.”
Scaling Data-Constrained Language Models (2023, neutral): “We note that our evaluation is in no means comprehensive and a larger benchmarking would be helpful [91, 64].”
Exploring the Potential of Large Language Models (LLMs)in Learning on Graphs (2023, background): “The former is one of the most popular lightweight deep text embedding models while the latter is the state-of-the-art model on the MTEB leaderboard [43].”
Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models (2025, neutral): “We explore three types of retrieval tasks: (1) Basic Relevance Retrieval, categorized into English, Chinese, and Multilingual, evaluated on MTEB (Muennighoff et al., 2023), CMTEB (Xiao et al., 2024), MMTEB (Enevoldsen et al., 2025), and MLDR (Chen et al., 2024), respectively; (2) Code Retrieval,…”
Improving Text Embeddings with Large Language Models (2023, methodology): “We evaluate the trained model on the MTEB benchmark [27].”
Text Embeddings by Weakly-Supervised Contrastive Pre-training (2022, methodology): “MTEB Benchmark [40] is recently proposed for benchmarking massive text embedding tasks.”
What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuning (2023, methodology): “Due to the superior performance of E5-Large-V2 in MTEB benchmark (Muennighoff et al., 2023), we speculate (Bhatia et al., 2023) the representation from the Model-based method not only encodes semantic information within the sentences but also reflects certain characteristics of the data for…”
Large Language Models for Information Retrieval: A Survey (2023, neutral): “This intriguing area remains open for further investigation and advancement.”
LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory (2024, positive): “For the retriever, we choose dense retrieval with the 1.5B Stella V5 model (Zhang, 2023), given its high performance on MTEB (Muennighoff et al., 2023).”
Towards General Text Embeddings with Multi-stage Contrastive Learning (2023, methodology): “Surprisingly, our model, pre-trained on this dataset, exhibits remarkable performance, surpassing BM25 and E5 model (Wang et al., 2022b) in zero-shot text retrieval tasks and surpassing many supervised models in the MTEB benchmark.”
LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders (2024, methodology): “We follow established practice from the masked language modeling literature and randomly mask a fraction of the tokens from the input sequence (Devlin et al., 2019; Liu et al., 2019).”
Tevatron Meets Megatron: Expert-Parallel LLM Reranker Training on an Academic Budget (2026, unknown): “Adding a suite beyond BEIR-15 (MTEB (Muennighoff et al., 2023), BRIGHT (SU et al., 2025), a private set) is one entry; the metric core, orchestrator, renderer, and backends are reused unchanged: BENCHMARKS = { "beir15": Benchmark(datasets=..., load_qrels=..., split_for=..., summary_layout=...), #…”
A Comparative Study on Affective Cues in Text Embeddings Across Psychological Emotion Theories (2026, unknown): “For a comprehensive analysis, we handpicked six open-weight and three proprietary models from the top entries of the Massive Text Embedding Benchmark (MTEB) 2 leaderboard curated by Hugging Face [14,35].”
Embedding World Knowledge into Tabular Models: Towards Best Practices for Embedding Pipeline Design (2026, unknown): “We selected a representative set of 16 LLM-based embedding models from the Massive Text Embedding Benchmark (MTEB) leaderboard [13] with ≤ 1 billion parameters, including all-MiniLM-L6-v2 [6], bge-large-en-v1.”
Geometry-Preserving Aggregation for Mixture-of-Experts Embedding Models (2026, unknown): “…dataset [1], which combines SNLI [19] and MultiNLI [20] sentence pairs for representation learning. b) Evaluation Data: Evaluation is performed on selected tasks from the Massive Text Embedding Benchmark (MTEB) [2]: • STSBenchmark: Semantic similarity evaluated using Spearman correlation [21].”
Evaluating Embedding Generalization: How LLMs, LoRA, and SLERP Shape Representational Geometry (2025, unknown): “[5 , 6] At the same time, the rise of very large language models (LLMs) as embedding backbones has created a new set of design choices.”
TurkEmbed: Turkish Embedding Model on Natural Language Inference & Sentence Text Similarity Tasks (2025, unknown): “The methodology involves combining diverse datasets with advanced training techniques, notably Matryoshka representation learning, and selecting base models from the MTEB leaderboard [14].”