RAVEN: A Dataset for Relational and Analogical Visual REasoNing

By Feng Gao, Baoxiong Jia, Yixin Zhu, Song-Chun Zhu. 2019

404 citations recorded by the source corpus.

Dramatic progress has been witnessed in basic vision tasks involving low-level perception, such as object recognition, detection, and tracking. Unfortunately, there is still enormous performance gap between artificial vision systems and human intelligence in terms of higher-level vision problems, especially ones involving reasoning. Earlier attempts in equipping machines with high-level reasoning have hovered around Visual Question Answering (VQA), one typical task associating vision and language understanding. In this work, we propose a new dataset, built in the context of Raven's Progressive Matrices (RPM) and aimed at lifting machine intelligence by associating vision with structural, relational, and analogical reasoning in a hierarchical representation. Unlike previous works in measuring abstract reasoning using RPM, we establish a semantic link between vision and reasoning by providing structure representation. This addition enables a new type of abstract reasoning by jointly operating on the structure representation. Machine reasoning ability using modern computer vision is evaluated in this newly proposed dataset. Additionally, we also provide human performance as a reference. Finally, we show consistent improvement across all models by incorporating a simple neural module that combines visual understanding and structure reasoning.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Molecular Biology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Molecular Biology

Citations in the corpus

Recent Advances of Continual Learning in Computer Vision: An Overview (2021, neutral): “Hayes and Kanan [101] proposed an analogical reasoning method, where Raven’s Progressive Matrices [102] are commonly used as the measurement method.”
Dark, Beyond Deep: A Paradigm Shift to Cognitive AI with Humanlike Common Sense (2020, methodology): “[374] with permission of the authors, c © 2019.”
Experts Collaboration Learning for Continual Multi-Modal Reasoning (2023, neutral): “tion [2], [7], [8], [9], [10], [11], [12].”
Selective Replay Enhances Learning in Online Continual Analogical Reasoning (2021, methodology): “While humans have strong abstract reasoning abilities, neural networks struggle with these problems [37, 38, 70, 72, 86].”
MMICL: Empowering Vision-language Model with Multi-Modal In-Context Learning (2023, methodology): “RAVEN (Zhang et al., 2019; Huang et al., 2023a) test is widely used to evaluate the nonverbal reasoning ability of VLMs.”
A neuro-vector-symbolic architecture for solving Raven’s progressive matrices (2022, background): “Equations (6) and (7) describe a VSA-based conjunctive formula grounded over the row and column being considered, respectively.”
What matters when building vision-language models? (2024, neutral): “TabMWP (Lu et al., 2023) 22,729 23,059 1,948,166 2.49% TAT-QA (Zhu et al., 2021) 2,199 13,215 283,776 2.18% HiTab (Cheng et al., 2022) 2,500 7,782 351,299 1.80% MultiHiertt (Zhao et al., 2022) 7,619 7,830 267,615 1.37% FinQA (Chen et al., 2021) 5,276 6,251 242,561 0.99% WikiSQL (Zhong et al., 2017) ”
The Abduction of Sherlock Holmes: A Dataset for Visual Abductive Reasoning (2022, background): “Visual understanding and reasoning that go beyond descriptive content have gained increasing attention, including work on visual and analogical reasoning [3, 21, 43, 77], scene graphs and semantics [23], commonsense interactions [49, 65], temporal/causal reasoning [71], and perceived importance [5].”
PTR: A Benchmark for Part-based Conceptual, Relational, and Physical Reasoning (2021, background): “RAVEN [62] associates vision with structural, relational, and analogical reasoning in a hierarchical representation.”
Trends in Integration of Vision and Language Research: A Survey of Tasks, Datasets, and Methods (2019, methodology): “35 ResNetDRT (Zhang et al., 2019) 59.”
R1-Onevision: Advancing Generalized Multimodal Reasoning Through Cross-Modal Formalization (2025, neutral): “With the advancement of model capabilities in visual reasoning, an increasing number of studies have proposed various benchmarks to evaluate the reasoning abilities of these models [12, 14, 36].”
Stratified Rule-Aware Network for Abstract Visual Reasoning (2020, positive): “We adopt the public implementations of LSTM, ResNet, and DRT in (Zhang et al. 2019a).”
Learning by Fixing: Solving Math Word Problems with Weak Supervision (2020, background): “Recently, there emerges various question-answering tasks that require human-like reasoning abilities (Qi et al. 2015; Tu et al. 2014; Zhang et al. 2019; Dua et al. 2019; Hong et al. 2019; Zhu et al. 2020; Zhang et al. 2020b; Li et al. 2020b; Yu et al. 2020).”