mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrieval

By Xin Zhang, Yanzhao Zhang, Dingkun Long, Wen Xie, Ziqi Dai, Jialong Tang, Huan Lin, Baosong Yang, Pengjun Xie, Fei Huang and 3 more. Conference on Empirical Methods in Natural Language Processing, 2024

342 citations recorded by the source corpus.

We present systematic efforts in building long-context multilingual text representation model (TRM) and reranker from scratch for text retrieval. We first introduce a text encoder (base size) enhanced with RoPE and unpadding, pre-trained in a native 8192-token context (longer than 512 of previous multilingual encoders). Then we construct a hybrid TRM and a cross-encoder reranker by contrastive learning. Evaluations show that our text encoder outperforms the same-sized previous state-of-the-art XLM-R. Meanwhile, our TRM and reranker match the performance of large-sized state-of-the-art BGE-M3 models and achieve better results on long-context retrieval benchmarks. Further analysis demonstrate that our proposed models exhibit higher efficiency during both training and inference. We believe their efficiency and effectiveness could benefit various researches and industrial applications.

Subfields: Artificial Intelligence, Information Systems

View Paper: https://www.semanticscholar.org/paper/a5d5a3a9b3c31d03c4ab863b01a18120b0a847d1

Citations in the corpus

• Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference (2024, neutral): “All models are trained by re-using the best hyper-parameters identified in Section 3.1.2.”
• Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models (2025, neutral): “For reranking, we compare with the rerankers of jina 1 , mGTE (Zhang et al., 2024b) and BGE-m3 (Chen et al., 2024).”
• SkMTEB: Slovak Massive Text Embedding Benchmark and Model Adaptation (2026, unknown): “(Zhang et al., 2024) snowflake-arctic-embed-l-v2.”
• TurkEmbed: Turkish Embedding Model on Natural Language Inference & Sentence Text Similarity Tasks (2025, unknown): “Similarly, the GTE-multilingual-base model [25], providing generalized embeddings suitable for various tasks across multiple languages, offers a versatile solution for Turkish NLP, particularly in cross-lingual applications.”
• The TellTail of Embeddings: Fingerprinting Retrievers in Black-Box Systems (2026, unknown): “5 , and gte-multi [43]; nomic-modernbert and nomic-v1.”
• Q2D-Web: A Large-Scale Benchmark for Retrieval in Agentic RAG Systems (2026, unknown): “…en arctic-l-v2 (Yu et al., 2024) dense 568M 1024 8192 multi bge-m3 (Chen et al., 2024a) dense 568M 1024 8192 multi e5-large-instruct (Wang et al., 2024) dense 560M 1024 512 multi gte-multilingual-base (Zhang et al., 2024) dense 305M 768 8192 multi mxbai-large-v1 (Lee et al., 2024; Li and Li, 2023)”
• Closing the Operational Gap in Semantic Caching (2026, unknown): “5 (Xiao et al., 2024) 0.660 0.372 0.288 0.095 0.564 GTE-ModernBERT-base (Zhang et al., 2024) 0.649 0.380 0.269 0.076 0.586 Jina-Embeddings-v2-base-en (Günther et al., 2024) 0.646 0.372 0.274 0.081 0.575 Nomic-embed-text-v1.”
• MLAIRE: Multilingual Language-Aware Information Retrieval Evaluation Protocal (2026, unknown): “…dense retrievers encompass diverse model lineages, ranging from widely adopted encoder-only families such as multilingual-e5 [32], bge-m3 [33], gte [34], snowflake-arctic [35], nomic-embed [36], embeddinggemma [37] and jina [38, 39], to recent LLM-based embedding models including Qwen3-Embedding…”
• ARTLAS: Mapping Art-Technology Institutions via Conceptual Axes, Text Embeddings, and Unsupervised Clustering (2026, unknown): “…(codebook 𝑘 ∈ { 5 , 7 , 9 , 11 , 13 } , UMAP 4D/8D, SVD, agglomerative/ 𝑘 - means, cluster 𝑘 ∈ { 8 , . . . , 15 } ): E5-large-v2 [27], GTE-Qwen2 [18], GTE-Qwen2+SBERT dual, GTE-Qwen2+BGE dual, SBERT-Mini+BGE dual, Nomic [32], BGE-M3 [33], SBERT-Mini [16], and baselines (Word2Vec [28],…”
• Efficient Temporal-aware Matryoshka Adaptation for Temporal Information Retrieval (2026, unknown): “5) (Zhang et al., 2024), bge-base-en-v1.”
• “Small” Large Language Models in the Hospital: Evaluation Study on Real-World Data in a Resource-Constrained Setting (2025, unknown): “Thus, we chose the following models for this evaluation: gte-multilingual-base [32], bge-m3 [33], and nomic-embed-text-v1.”
• mmBERT: A Modern Multilingual Encoder with Annealed Language Learning (2025, unknown): “Baselines We use a variety of baselines: the older but still ubiquitous XLM-R (Conneau et al., 2019), mGTE (Zhang et al., 2024), and EuroBERT-210m (Boizard et al., 2025) for the base size and mDistilBERT (Sanh et al., 2019) and Multilingual MiniLM (Wang et al., 2020) for the small size.”
• PyLate: Flexible Training and Retrieval for Late Interaction Models (2025, unknown): “The single vector paradigm facilitates pre-computation and indexing of document embeddings, enabling rapid retrieval by identifying nearest neighbors to a query ∗ Equal contribution.”