InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models

By Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Yuchen Duan, Hao Tian, Weijie Su, Jie Shao and 38 more. arXiv.org, 2025

1,590 citations recorded by the source corpus.

We introduce InternVL3, a significant advancement in the InternVL series featuring a native multimodal pre-training paradigm. Rather than adapting a text-only large language model (LLM) into a multimodal large language model (MLLM) that supports visual inputs, InternVL3 jointly acquires multimodal and linguistic capabilities from both diverse multimodal data and pure-text corpora during a single pre-training stage. This unified training paradigm effectively addresses the complexities and alignment challenges commonly encountered in conventional post-hoc training pipelines for MLLMs. To further improve performance and scalability, InternVL3 incorporates variable visual position encoding (V2PE) to support extended multimodal contexts, employs advanced post-training techniques such as supervised fine-tuning (SFT) and mixed preference optimization (MPO), and adopts test-time scaling strategies alongside an optimized training infrastructure. Extensive empirical evaluations demonstrate that InternVL3 delivers superior performance across a wide range of multi-modal tasks. In particular, InternVL3-78B achieves a score of 72.2 on the MMMU benchmark, setting a new state-of-the-art among open-source MLLMs. Its capabilities remain highly competitive with leading proprietary models, including ChatGPT-4o, Claude 3.5 Sonnet, and Gemini 2.5 Pro, while also maintaining strong pure-language proficiency. In pursuit of open-science principles, we will publicly release both the training data and model weights to foster further research and development in next-generation MLLMs.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/cddf14e5b97090111d3fa814c9aec60e2bf24b8a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Tong Lu
2. Zhaoyang Liu
3. Weijie Su
4. Dahua Lin
5. Yinan He
6. Conghui He
7. Xizhou Zhu
8. Haodong Duan

In Computer Vision and Pattern Recognition

1. Tong Lu
2. Luc Van Gool
3. Xizhou Zhu
4. Dahua Lin
5. Dacheng Tao
6. Yu Qiao
7. Xiaogang Wang
8. Chunhua Shen

Citations in the corpus

• InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency (2025, positive): “…corpora can be classified into two categories: (1) Multimodal data: this subset of data is mainly sourced from the training corpora of InternVL3 [187], covering a diverse range of domains such as image captioning, general question answering, mathematics, scientific disciplines, charts, optical…”
• LITTA: Late-Interaction and Test-Time Alignment for Visually-Grounded Multimodal Retrieval (2026, unknown): “Parallel progress in long-document and multi-page understanding further highlights the need for robust evidence localization under complex layouts and long contexts [15, 38, 42, 9].”
• mKG-RAG: Leveraging Multimodal Knowledge Graphs in Retrieval-Augmented Generation for Knowledge-intensive VQA (2025, unknown): “In Table 6, we provide a detailed comparison of VQA scores across different MLLMs with varying parameter sizes, including Phi3V [1], InternVL3 [69], LLaMA-3.”
• VaRS-Doc: Interpretation-Aware Variant Representations via Latent Self-Probing for Visual Document Retrieval (2026, unknown): “Recent MLLMs, such as LLaVA (Liu et al. 2023), PaliGemma (Beyer et al. 2024), Qwen-VL (Bai et al. 2023), and InternVL (Chen et al. 2024b; Zhu et al. 2025), further strengthen visual-textual alignment and reasoning over text-rich images.”
• LightSTAR: Efficient Visual Document Retrieval via Lightweight Selection with Vision-Adaptive Refinement (2026, unknown): “For Vision-adaptive Semantic Refinement, we build upon InternVL3 [53] as the vision-language backbone.”