Qwen-VL: A Frontier Large Vision-Language Model with Versatile Abilities

By Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, Jingren Zhou. 2023

1,892 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Large Language Models, Multimodal Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Large Language Models

In Multimodal Learning

Citations in the corpus

Improved Baselines with Visual Instruction Tuning (2023, methodology): “5 achieves the best overall performance on 12 benchmarks, despite using magnitudes smaller pretraining and instruction tuning data compared with other methods [3, 13].”
Evaluating Object Hallucination in Large Vision-Language Models (2023, background): “Considering their impressive performance on complex vision-language tasks (Chen et al., 2023; Bai et al., 2023; Li et al., 2023a), it is counter-intuitive that the hallucination problem of LVLMs is so severe.”
Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models (2025, positive): “…that facilitates the generation of high-quality CoT processes, which are post-processed and integrated with the original data to create the final Vision-R1-cold dataset. to progressively develop more complex reasoning processes while effectively guiding MLLMs toward enhanced reasoning capability.”
Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning (2024, methodology): “In terms of training strategy, recent works [34, 3, 60, 80, 7, 38] commonly employ a 2-stage framework.”
Hallucination of Multimodal Large Language Models: A Survey (2024, positive): “Qwen-VL [2] has shown the effectiveness of gradually enlarging image resolution from 224 × 224 to 448 × 448.”
RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-Grained Correctional Human Feedback (2023, methodology): “Models are pretrained on large-scale image-text pairs [4, 11, 19, 42, 47] or interleaved data [2, 3, 16] to learn the semantic mapping between visual and text signals.”
Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generation (2024, neutral): “5 ( 7 B) [48] and Qwen-VL-Chat ( 7 B) [3] .”
ShareGPT4Video: Improving Video Understanding and Generation with Better Captions (2024, background): “The vision-language community has recently made significant advancements in the field of large image-language models [36, 35, 37, 10, 13, 11, 58, 59, 2, 72, 33].”
Intern VL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks (2023, methodology): “Vision large language models (VLLMs) [3, 5, 19, 21, 28, 69, 87, 113, 147], which leverage LLMs, have also achieved significant breakthroughs, enabling sophisticated vision-language dialogues and interactions.”
GLaMM: Pixel Grounding Large Multimodal Model (2023, neutral): “LLM LLaVA W BLIP-2 [24] Vicuna-13B 38.1 InstructBLIP [6] Vicuna-7B 60.9 Qwen-VL [3] Qwen-7B 63.4 Qwen-VL-Chat [3] Qwen-7B 58.6 LLaVA-1.”
Hallusionbench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Models (2023, neutral): “5 [32], Gemini Pro Vision [39], Claude 3 [4], MiniGPT4 [63], MiniGPT5 [62], GiT [44], InstructBLIP [12], Qwen-VL [7], mPLUG-Owl-v1 [50], mPLUG-Owl-v2 [51], LRV-Instruction [29], BLIP2 [22], BLIP2-T5 [22], and Open-Flamingo [3].”
Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perception (2024, background): “…(MLLM) Liu et al. [2023d], Zhu et al. [2023], Ye et al. [2023a], Dai et al. [2023], Liu et al. [2023e], Chen et al. [2023], Ye et al. [2023b], Bai et al. [2023], Lin et al. [2023] rapidly progress and exhibit remarkably visual comprehension capabilities, the realization of MLLM-based agents…”
MMBench: Is Your Multi-modal Model an All-around Player? (2023, methodology): “In the realm of proprietary models, the APIs of multiple powerful VLMs have also been made publicly available to prosper downstream applications, including GPT-4v [37], Gemini-Pro-V [44], and Qwen-VL-Max [6].”
Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding (2023, methodology): “5 [44, 45], InstructBLIP [12], and Qwen-VL [5].”
mPLUG-Owl3: Towards Long Image-Sequence Understanding in Multi-Modal Large Language Models (2024, methodology): “It also matches or surpasses the performance of larger models such as CogVLM (Wang et al., 2023) and EVLM-Chat (Chen et al., 2024b). mPLUG-Owl3 does not achieve state-of-the-art performance on the AI2D dataset.”
Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Models (2023, methodology): “…GPT4 (OpenAI, 2023), Flamingo (Alayrac et al., 2022), Kosmos (Peng et al., 2023), BLIP (Li et al., 2022), Shikra (Chen et al., 2023), Emu (Sun et al., 2023) and Qwen-VL (Bai et al., 2023) have proposed different integration method to enable image understanding or generation capabilities for LLMs.”
Qwen2 Technical Report (2024, methodology): “In the meantime, we have unveiled the vision-language model Qwen-VL (Bai et al., 2023b), and launched the audio-language model Qwen-Audio (Chu et al., 2023).”
VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs (2024, methodology): “The video-text sources include Panda-70M (Chen et al., 2024b), VIDAL-10M (Zhu et al., 2023a) WebVid-10M (Bain et al., 2021), and InternVid-10M (Wang et al., 2023), while the image-text sources include CC-3M (Changpinyo et al., 2021) and DCI (Urbanek et al., 2023).”