LVBench: An Extreme Long Video Understanding Benchmark

By Weihan Wang, Zehai He, Wenyi Hong, Yean Cheng, Xiaohan Zhang, Ji Qi, Shiyu Huang, Bin Xu, Yuxiao Dong, Ming Ding and 1 more. IEEE International Conference on Computer Vision, 2024

399 citations recorded by the source corpus.

Recent progress in multimodal large language models has markedly enhanced the understanding of short videos (typically under one minute), and several evaluation datasets have emerged accordingly. However, these advancements fall short of meeting the demands of real-world applications such as embodied intelligence for long-term decision-making, in-depth movie reviews and discussions, and live sports commentary, all of which require comprehension of long videos spanning several hours. To address this gap, we introduce LVBench, a benchmark specifically designed for long video understanding. Our dataset comprises publicly sourced videos and encompasses a diverse set of tasks aimed at long video comprehension and information extraction. LVBench is designed to challenge multimodal models to demonstrate long-term memory and extended comprehension capabilities. Our extensive evaluations reveal that current multimodal models still underperform on these demanding long video understanding tasks. Through LVBench, we aim to spur the development of more advanced models capable of tackling the complexities of long video comprehension.

Subfields: Control and Systems Engineering

View Paper: https://www.semanticscholar.org/paper/74e0f1aa5aa3bda5991e421a9d168890ac7556fa

Citations in the corpus

• Qwen3-Omni Technical Report (2025, neutral): “To evaluate performance on dynamic visual data, we report results on three long video understanding benchmarks: Video-MME (Fu et al., 2024), LVBench (Wang et al., 2024b), and MLVU (Zhou et al., 2025a).”
• GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning (2025, neutral): “…Some [18], Webquest-QA [56]; • Coding : Design2Code [43], Flame-React-Eval [1]; • Spatial Reco & Reasoning : OminiSpatial [24], CV-Bench [51], ERQA [49], All-Angles Bench [65]; • Video Understanding : VideoMME [10], MMVU [71], VideoMMMU [22], LVBench [58], MotionBench [19], MVBench [28]; Setting.”
• Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis (2024, neutral): “Due to the inherently temporal characteristics of video modalities, specialized benchmarks have been created to evaluate temporal comprehension, underscoring the current limitations of video MLLMs in understanding video content [3, 15, 32, 41, 57, 62].”
• InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency (2025, neutral): “Furthermore, the growth of multimodal capabilities, e.g., long visual context and high-resolution understanding [5, 81, 140, 147, 180, 187], often comes with ever increasing computational costs, which have become a crucial bottleneck of real-world applications.”
• Qwen3-VL Technical Report (2025, neutral): “We conduct a comprehensive evaluation across a diverse set of video understanding tasks, encompassing general video understanding (VideoMME (Fu et al., 2024a), MVBench (Li et al., 2024b)), temporal video grounding (Charades-STA (Gao et al., 2017)), video reasoning (VideoMMMU (Hu et al., 2025), MMVU ”
• VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understanding (2025, neutral): “…benchmarks: (1) MLVU [131]: diverse long-video understanding tasks for videos ranging from 3 minutes to more than 2 hours, (2) LongVideoBench [132]: video reasoning over the referred context within long video-language interleaved inputs, and (3) LVBench [133]: extreme long video understanding.”