M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation

By Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, Zheng Liu. Annual Meeting of the Association for Computational Linguistics, 2024

1,560 citations recorded by the source corpus.

In this paper, we introduce a new embedding model called M3-Embedding, which is distinguished for its versatility in \textit{Multi-Linguality}, \textit{Multi-Functionality}, and \textit{Multi-Granularity}. It provides a uniform support for the semantic retrieval of more than 100 working languages. It can simultaneously accomplish the three common retrieval functionalities: dense retrieval, multi-vector retrieval, and sparse retrieval. Besides, it is also capable of processing inputs of different granularities, spanning from short sentences to long documents of up to 8,192 tokens. The effective training of M3-Embedding presents a series of technical contributions. Notably, we propose a novel self-knowledge distillation approach, where the relevance scores from different retrieval functionalities can be integrated as the teacher signal to enhance the training quality. We also optimize the batching strategy, which enables a large batch size and high training throughput to improve the discriminativeness of embeddings. M3-Embedding exhibits a superior performance in our experiment, leading to new state-of-the-art results on multilingual, cross-lingual, and long-document retrieval benchmarks.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/677f89b61eff0f5ad3ae10c2b9208aaa16aa4239

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Liang Wang
2. Vojtěch Vančura
3. Yanzhao Zhang
4. Ziyan Jiang
5. Xin Zhang
6. Pavel Kordík
7. Nan Yang
8. Wenhu Chen

Citations in the corpus

• LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Models (2024, neutral): “To balance data collection costs and user evaluation, we aim to gather about 500 questions monthly, selecting 100-300 for our final L IVE B ENCH problem set, tagged with identifiers like LiveBench-2024-05 .”
• Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference (2024, neutral): “Models are trained on 810k samples from MS-Marco (Bajaj et al., 2016) and teacher scores from BGE-M3 (Chen et al., 2024), using the PyLate library (Chaffin and Sourty, 2024).”
• mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrieval (2024, methodology): “Sparse Representation Chen et al. (2024) show that neural sparse representations (term/token weights predicted by TRM) could greatly improve the long-context retrieval performance.”
• Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models (2025, neutral): “…Chinese, and Multilingual, evaluated on MTEB (Muennighoff et al., 2023), CMTEB (Xiao et al., 2024), MMTEB (Enevoldsen et al., 2025), and MLDR (Chen et al., 2024), respectively; (2) Code Retrieval, evaluated on MTEB-Code (Enevoldsen et al., 2025), which comprises only code-related retrieval…”
• What Makes a Good Fiqh Retriever? Answer Retrieval for Arabic Islamic Jurisprudence (2026, unknown): “The multilingual category includes BGE-M3 (Chen et al., 2024), Multilingual-E5-large-instruct (Wang et al., 2024), Arctic-Embed-L v2.”
• A Comparative Study on Affective Cues in Text Embeddings Across Psychological Emotion Theories (2026, unknown): “The advent of text encoders has transformed the way of converting textual content into numerical representations, known as embeddings, which now serve as core components for a variety of tasks, e.g., semantic similarity, retrieval, and reranking [8].”
• Temporal Preference Optimization for Unsupervised Retrieval (2026, unknown): “GTE (Li et al., 2023) improves generalization by training on diverse datasets, while M3-Embedding (Chen et al., 2024) uses self-distillation to unify signals from multiple retrieval paradigms.”
• IdioLink: Retrieving Meaning Beyond Words Across Idiomatic and Literal Expressions (2026, unknown): “5 (Qwen et al., 2025), the Gemma-based BGE model (Chen et al., 2024), and the Qwen3 Embedding models (Zhang et al., 2025c).”
• Dtour: a Steerable Tour De Vis through High-dimensional Data (2026, unknown): “To compare embedding models, we construct a sequential tour from 3M arXiv title-abstract embeddings produced by four models spanning 2023– 2026: SPECTER2 [35], BGE-M3 [11], Nomic Embed Text v2 [30], and F2LLM-v2-8B [47] (top-ranked on clustering benchmarks), each reduced to 2D with default UMAP.”
• Contrastive Learning Using Graph Embeddings for Domain Adaptation of Language Models in the Process Industry (2025, unknown): “We also aim to optimize the methodology by initializing the GE model with high-performing text encoders designed for semantic text similarity tasks, such as M3 (Chen et al., 2024).”
• Chunk Twice, Embed Once: A Systematic Study of Segmentation and Representation Trade-offs in Chemistry-Aware Retrieval-Augmented Generation (2025, unknown): “The multitask bge-m3 model 5 supports several embedding and retrieval settings [59].”
• NEAR: Neighborhood-Aware Residuals for Memory Retrieval (2026, unknown): “We evaluate retrieval on eight benchmarks: LoCoMo (Maharana et al., 2024), Long-MemEval (Wu et al., 2025), and REALTALK (Lee et al., 2025) for conversational memory, and ESG-Reports (Macé et al., 2025), QASPER (Dasigi et al., 2021), PeerQA (Baumgärtner et al., 2025), CovidQA (Möller et al., 2020), a”
• The TellTail of Embeddings: Fingerprinting Retrievers in Black-Box Systems (2026, unknown): “We use BAAI/bge-reranker-v2-m3 [62], [63], the reranker OpenWebUI suggests when enabling reranking.”
• Embedding Models Measure in Peculiar Ways (2026, unknown): “5 (Xiao et al., 2024) and its multilingual successor bge-m3 (Chen et al., 2024); mxbai-embed-large-v1 (Lee et al., 2024), which is trained with the AnglE objective (Li and Li, 2024); and the first release of the multilingual Granite embedding models ( granite-embedding-107m-multilingual and…”
• Benchmarking Embedding Models for ESG Data (2026, unknown): “• bge-m3 : A multilingual embedding model from the BAAI General Embedding (BGE) family, supporting cross-lingual retrieval and particularly relevant for multilingual ESG data [38].”