ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning

By Ahmed Masry, Do Xuan Long, J. Tan, Shafiq R. Joty, Enamul Hoque. Findings, 2022

1,654 citations recorded by the source corpus.

Charts are very popular for analyzing data. When exploring charts, people often ask a variety of complex reasoning questions that involve several logical and arithmetic operations. They also commonly refer to visual features of a chart in their questions. However, most existing datasets do not focus on such complex reasoning questions as their questions are template-based and answers come from a fixed-vocabulary. In this work, we present a large-scale benchmark covering 9.6K human-written questions as well as 23.1K questions generated from human-written chart summaries. To address the unique challenges in our benchmark involving visual and logical reasoning over charts, we present two transformer-based models that combine visual features and the data table of the chart in a unified way to answer questions. While our models achieve the state-of-the-art results on the previous datasets as well as on our benchmark, the evaluation also reveals several challenges in answering complex reasoning questions.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Signal Processing

View Paper: https://www.semanticscholar.org/paper/b611c501269224702d1a9942c8600a31ec66ab28

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

Hallusionbench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Models (2023, background): “These models have demonstrated profound capabilities in various applications and significantly enhance the performance in image reasoning tasks [5, 18, 20, 30, 31, 36, 38, 42, 47].”
Qwen3-VL Technical Report (2025, neutral): “…across a diverse range OCR related benchmarks: DocVQA (Mathew et al., 2021b), InfoVQA (Mathew et al., 2021a), AI2D (Kembhavi et al., 2016), ChartQA (Masry et al., 2022), OCRBench (Liu et al., 2024), OCRBench_v2 (Fu et al., 2024b), CC-OCR (Yang et al., 2024b), OmniDocBench (Ouyang et al., 2024),…”
Qwen2.5-Omni Technical Report (2025, neutral): “5-Omni on various OCR benchmarks, such as AI2D (Kembhavi et al., 2016), TextVQA (Singh et al., 2019), DocVQA (Mathew et al., 2021), ChartQA (Masry et al., 2022), and OCRBench_v2 (Fu et al., 2024b).”
Gemma 3 Technical Report (2025, neutral): “AI2D (Kembhavi et al., 2016), ChartQA (Masry et al., 2022), VQA v2 (Goyal et al., 2017), BLINK (Fu et al., 2024), OK-VQA (Marino et al., 2019), TallyQA (Acharya et al., 2018), Spa-tialSense VQA (Yang et al., 2019), CountBench VQA (Paiss et al., 2023).”
Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs (2024, methodology): “We use a suite of commonly used benchmarks [19, 39, 47, 50, 76, 77, 84, 85, 89, 112, 117, 128, 134], which is the intersection of those used in recent MLLM research [68, 70, 128].”
Mini-Gemini: Mining the Potential of Multi-Modality Vision Language Models (2024, neutral): “For illustrative clarity, we consider the concurrent processing of both modalities.”
PaLI-X: On Scaling up a Multilingual Vision and Language Model (2023, methodology): “Benchmark datasets The Image Captioning and VQA benchmarks used for evaluation is summarized in Appendix B, including 6 Image Captioning benchmarks (COCO (Karpathy split [30]), NoCaps [31], TextCaps [32], VizWiz-Cap [33], Screen2Words [34], Widget-Cap [35]) and 13 VQA benchmarks (VQAv2 [36], OKVQA [”
How far are we to GPT-4V? Closing the gap to commercial multimodal models with open-source suites (2024, neutral): “The training of InternVL 1.5 was divided into two stages.”
Emu3: Next-Token Prediction is All You Need (2024, methodology): “OCRBench [59](with normalized results), MMVet [98], POPE [51], VQAv2 [27], GQA [34], TextVQA [78], ChartQA [61], AI2D [36], RealWorldQA [91], MMMU [99], and MMbench [58].”
Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling (2024, neutral): “ChartQA [181]: ChartQA is a dataset focused on assessing models’ abilities to interpret and reason with data visualizations such as charts and graphs.”
InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency (2025, neutral): “…the comprehensive capabilities of the model across tasks related to text, document, and chart comprehension, we conduct an extensive assessment on nine benchmarks: AI2D [55], ChartQA [87], TextVQA [112], DocVQA [89], InfoVQA [88], OCRBench [72], SEED-2-Plus [59], CharXiv [148], and VCR [176].”
Intern VL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks (2023, neutral): “…588K COCO Caption [20], TextCaps [96] VQAv2 [40], OKVQA [79], A-OKVQA [93], VQA 1.1M IconQA [75], AI2D [53], GQA [48] OCR-VQA [82], ChartQA [80], DocVQA [25], ST-VQA [11], EST-VQA [116], InfoVQA [81] model can generate a feature map F ∈ R H/ 14 × W/ 14 × D for dense prediction tasks, or…”
LLaVA-OneVision: Easy Visual Task Transfer (2024, neutral): “In our development, we evaluate on AI2D [54], ChartQA [101], DocVQA [103], and InfoVQA [102] to examine the model’s fine-grained understanding of tables, charts, and diagrams, as well as MME [28] for formatting control, since it requires only Yes or No answers.”
VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understanding (2025, neutral): “…DocVQA [113] for document understanding, which evaluates the model’s ability to process and extract information from text in documents; 2) ChartQA [42] and InfoVQA [113] for chart understanding, assessing the model’s ability to interpret and reason about data presented in graphical forms such as…”
OCRBench: on the hidden mystery of OCR in large multimodal models (2023, neutral): “We assess these LMMs on DocVQA[55], InfographicVQA [56] and ChartQA [57].”
Qwen-VL: A Frontier Large Vision-Language Model with Versatile Abilities (2023, neutral): “We assess our models’ ability toward text-oriented visual question answering on several benchmarks includingTextVQA(Sidorov et al., 2020), DocVQA (Mathew et al., 2021), ChartQA (Masry et al., 2022), AI2Diagram (Kembhavi et al., 2016), and OCR-VQA (Mishra et al., 2019).”
R1-VL: Learning to Reason with Multimodal Large Language Models via Step-Wise Group Relative Policy Optimization (2025, neutral): “For evaluation, we adopt 8 widely-used multimodal benchmarks for comprehensively evaluating our proposed StepGRPO, including MathVista [21], MM-Star [5], Math-Vision [36], ChartQA [24], DynaMath [51], HallusionBench [11], MathVerse [48] and MME [10].”