MiniCPM-V: A GPT-4V Level MLLM on Your Phone
By Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He and 13 more. arXiv.org, 2024
1,205 citations recorded by the source corpus.
The recent surge of Multimodal Large Language Models (MLLMs) has fundamentally reshaped the landscape of AI research and industry, shedding light on a promising path toward the next AI milestone. However, significant challenges remain preventing MLLMs from being practical in real-world applications. The most notable challenge comes from the huge cost of running an MLLM with a massive number of parameters and extensive computation. As a result, most MLLMs need to be deployed on high-performing cloud servers, which greatly limits their application scopes such as mobile, offline, energy-sensitive, and privacy-protective scenarios. In this work, we present MiniCPM-V, a series of efficient MLLMs deployable on end-side devices. By integrating the latest MLLM techniques in architecture, pretraining and alignment, the latest MiniCPM-Llama3-V 2.5 has several notable features: (1) Strong performance, outperforming GPT-4V-1106, Gemini Pro and Claude 3 on OpenCompass, a comprehensive evaluation over 11 popular benchmarks, (2) strong OCR capability and 1.8M pixel high-resolution image perception at any aspect ratio, (3) trustworthy behavior with low hallucination rates, (4) multilingual support for 30+ languages, and (5) efficient deployment on mobile phones. More importantly, MiniCPM-V can be viewed as a representative example of a promising trend: The model sizes for achieving usable (e.g., GPT-4V) level performance are rapidly decreasing, along with the fast growth of end-side computation capacity. This jointly shows that GPT-4V level MLLMs deployed on end devices are becoming increasingly possible, unlocking a wider spectrum of real-world AI applications in the near future.
Subfields: Artificial Intelligence, Computer Networks and Communications
View Paper: https://www.semanticscholar.org/paper/bd0fae435b93819d407f36d9d52565d5dfb5f87c
Citations in the corpus
- LlaVA-CoT: Let Vision Language Models Reason Step-By-Step (2024, neutral): “6-8B [62], Llama-3.”
- A survey on multimodal large language models (2023, neutral): “To handle these complex requirements, some hand-crafted in-context (see §3.1) examples are used as references [75,76,80].”
- Qwen2.5-Omni Technical Report (2025, neutral): “MiniCPM-o (Yao et al., 2024) 0.524 Qwen-Audio (Chu et al., 2023a) 0.557 Qwen2-Audio (Chu et al., 2024a) 0.553 Qwen2.”
- mPLUG-Owl3: Towards Long Image-Sequence Understanding in Multi-Modal Large Language Models (2024, neutral): “5 (Yao et al., 2024).”
- Fix Before Search: Benchmarking Agentic Query Visual Pre-processing in Multimodal Retrieval-augmented Generation (2026, unknown): “We sample the same 500 examples as Image-to-Image Retrieval paradigm, but now index both the image and its associated Wikipedia article title as a multimodal unit using GME (Zhang et al., 2024).”
- Language-Agnostic Visual Embeddings for Cross-Script Handwriting Retrieval (2026, unknown): “General Visual-Language Models With the rapid evolution of deep learning, General Visual-Language Models (GVLMs) [ Abdin et al. , 2024; AI, 2024; Yao et al. , 2024; Chen et al. , 2025; Bai et al. , 2025; Wang et al. , 2025 ] have exhibited powerful capabilities in handwriting understanding driven…”