FinQA: A Dataset of Numerical Reasoning over Financial Data

By Wenhu Chen, Charese Smiley, Sameena Shah, Iana Borova, Dylan Langdon, Reema Moussa, Matthew I. Beane, Ting-Hao 'Kenneth' Huang, Bryan R. Routledge, William Yang Wang. Conference on Empirical Methods in Natural Language Processing, 2021

739 citations recorded by the source corpus.

The sheer volume of financial statements makes it difficult for humans to access and analyze a business’s financials. Robust numerical reasoning likewise faces unique challenges in this domain. In this work, we focus on answering deep questions over financial data, aiming to automate the analysis of a large corpus of financial documents. In contrast to existing tasks on general domain, the finance domain includes complex numerical reasoning and understanding of heterogeneous representations. To facilitate analytical progress, we propose a new large-scale dataset, FinQA, with Question-Answering pairs over Financial reports, written by financial experts. We also annotate the gold reasoning programs to ensure full explainability. We further introduce baselines and conduct comprehensive experiments in our dataset. The results demonstrate that popular, large, pre-trained models fall far short of expert humans in acquiring finance knowledge and in complex multi-step numerical reasoning on that knowledge. Our dataset – the first of its kind – should therefore enable significant, new community research into complex application domains. The dataset and code are publicly available at https://github.com/czyssrs/FinQA.

Subfields: Artificial Intelligence, Management Science and Operations Research

View Paper: https://www.semanticscholar.org/paper/99053e3a708fc27709c9dab33110dc98b187c158

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

Citations in the corpus

BloombergGPT: A Large Language Model for Finance (2023, background): “Financial NLP tasks (Shah et al., 2022) include sentiment analysis (Araci, 2019), named entity recognition (Salinas Alvarado et al., 2015), news classification (Sinha and Khandait, 2020), and question answering (Chen et al., 2021b, 2022).”
Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoning (2022, neutral): “Note that 77.6% of questions in TAT-QA can be solvable without mathematical reasoning and 50.0% of questions in FinQA are not table-must to be answered.”
Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks (2022, methodology): “Dataset Split Example Domain Input Output GSM8K Cobbe et al. (2021) Test 1318 MWP Question Number AQuA Ling et al. (2017) Test 253 MWP Question Option SVAMP Patel et al. (2021) Test 1000 MWP Question Number MultiArith Roy & Roth (2015) Test 600 MWP Question Number TabMWP Lu et al. (2022) Test 7861 M”
Large Language Models are few(1)-shot Table Reasoners (2022, background): “We also report results for Codex (Chen et al., 2021a) (Davinci-code-002) on some datasets.”
What matters when building vision-language models? (2024, neutral): “TabMWP (Lu et al., 2023) 22,729 23,059 1,948,166 2.49% TAT-QA (Zhu et al., 2021) 2,199 13,215 283,776 2.18% HiTab (Cheng et al., 2022) 2,500 7,782 351,299 1.80% MultiHiertt (Zhao et al., 2022) 7,619 7,830 267,615 1.37% FinQA (Chen et al., 2021) 5,276 6,251 242,561 0.99% WikiSQL (Zhong et al., 2017) ”
Learning to Imagine: Integrating Counterfactual Thinking in Neural Discrete Reasoning (2022, background): “As to extending the context, several studies try to enable the NDR model to operate on context with semi-structured tabular data and hybrid data (Chen et al., 2020c; Herzig et al., 2020; Chen et al., 2021).”
FinGPT: Open-Source Financial Large Language Models (2023, neutral): “Other noteworthy datasets include FinRED [Sharma et al. , 2022], instrumental for information extraction tasks, FINQA [Chen et al. , 2021] for numerical reasoning assessments, and FinRAD [Ghosh et al. , 2021], which is crucial for understanding and identifying financial terms.”
TheoremQA: A Theorem-driven Question Answering dataset (2023, methodology): “…University GMAT/GRE NO MathQA (Amini et al., 2019) Arithm/Algebra University AQuA NO MathQA-Python (Austin et al., 2021) Arithm/Algebra University AQuA NO FinQA (Chen et al., 2021c) Finance University CrowdSource NO TAT-QA (Zhu et al., 2021) Finance University CrowdSource NO on our dataset.”
A Survey of Deep Learning for Mathematical Reasoning (2022, neutral): “…this gap, datasets annotated with intermediate rationales such as logic forms (Tafjord et 2019; Lu et al., 2021a), programs (Amini et al., 2019; Chen et al., 2021c,a; Cao and Xiao, 2022; Chen et al., 2022a), and reasoning graphs (Zhang et al., 2021) have been proposed to train models for…”
Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs (2023, methodology): “In our benchmark, we present 8 basic math operations, including 4 complex operations as [7]: MINUS , ADDITION , DIVISION , MULTIPLY .”
Benchmarking Embedding Models for ESG Data (2026, unknown): “In the financial and ESG context, recent work by [19] explored the application of RAG systems to financial question answering, high-lighting the importance of high-quality retrieval components for domain-specific applications.”
Granite Embedding R2 Models (2025, unknown): “• OTT-QA: Open domain table-and-text question answering dataset requiring retrieval and fusion of both tabular data and textual passages from large pools.”