Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling

By Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu and 30 more. arXiv.org, 2024

1,631 citations recorded by the source corpus.

We introduce InternVL 2.5, an advanced multimodal large language model (MLLM) series that builds upon InternVL 2.0, maintaining its core model architecture while introducing significant enhancements in training and testing strategies as well as data quality. In this work, we delve into the relationship between model scaling and performance, systematically exploring the performance trends in vision encoders, language models, dataset sizes, and test-time configurations. Through extensive evaluations on a wide range of benchmarks, including multi-discipline reasoning, document understanding, multi-image / video understanding, real-world comprehension, multimodal hallucination detection, visual grounding, multilingual capabilities, and pure language processing, InternVL 2.5 exhibits competitive performance, rivaling leading commercial models such as GPT-4o and Claude-3.5-Sonnet. Notably, our model is the first open-source MLLMs to surpass 70% on the MMMU benchmark, achieving a 3.7-point improvement through Chain-of-Thought (CoT) reasoning and showcasing strong potential for test-time scaling. We hope this model contributes to the open-source community by setting new standards for developing and applying multimodal AI systems. HuggingFace demo see https://huggingface.co/spaces/OpenGVLab/InternVL

Subfields: Artificial Intelligence, Information Systems and Management

View Paper: https://www.semanticscholar.org/paper/5f49ec9560ca9e03eff32a607f6caabc08f98926

Citations in the corpus

• VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model (2025, neutral): “To-day, some of the most widely adopted open-source VLM series include LLaVA[25, 30, 31], QwenVL[6, 50], and In-ternVL [10, 11].”
• InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency (2025, neutral): “During the SFT phase, we adopt the same objective as in the pre-training stage and use the square averaging strategy [13] to calculate the final loss.”
• InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models (2025, neutral): “…bridging modality gaps often necessitates incorporating auxiliary data from specialized domains (e.g., optical character recognition scenarios) and intricate parameter-freezing or multi-stage fine-tuning schedules to ensure that core linguistic capacities remain uncompromised [73, 7, 5, 18].”
• VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learning (2025, neutral): “5-78B [Chen et al., 2024b], Llava-Onevision [Li et al., 2024], Llama-4-Scout and Kimi-VL [Team et al., 2025].”
• SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics (2025, neutral): “…other modalities, sparking interest towards multi-modal vision-language (VLMs) (Alayrac et al., 2022; Chen et al., 2023; Huang et al., 2023; Liu et al., 2023b; Chen et al., 2024; Shukor et al., 2023b) and audio-language models (ALMs) (Défossez et al., 2024; Das et al., 2024; Borsos et al., 2023).”
• VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understanding (2025, neutral): “Building upon the success of LLMs, Multimodal LLMs (MLLMs) [7–10] have been proposed for multimodal understanding.”
• AgiBot World Colosseo: A Large-Scale Manipulation Platform for Scalable and Intelligent Embodied Systems (2025, neutral): “To broaden the data pool by incorporating internet-scale human videos lacking action labels and cross-embodiment robot data, we employ latent actions [30] in Stage 1 to model the inverse dynamics of consecutive frames.”
• Emerging Properties in Unified Multimodal Pretraining (2025, neutral): “5 [12], demonstrating that our MoT design effectively mitigates task conflicts while maintaining strong visual understanding capabilities.”
• R1-Onevision: Advancing Generalized Multimodal Reasoning Through Cross-Modal Formalization (2025, neutral): “[2, 4, 18, 26] achieve excellent visual understanding ability by integrating both visual and textual data.”