Comparing machines and humans on a visual categorization test

By Franccois Fleuret, Ting Li, Charles Dubout, E. Wampler, Steven Yantis, Donald Geman. 2011

180 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Cognitive Science & Psychology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Cognitive Science & Psychology

Citations in the corpus

A Study and Comparison of Human and Deep Learning Recognition Performance under Visual Distortions (2017, neutral): “[7] test human and computer performance”
On the Binding Problem in Artificial Neural Networks (2020, background): “(d) Neural networks also struggle to learn visual relations such as whether two shapes are the same or different (Fleuret et al., 2011; Kim et al., 2018).”
Knowledge Matters: Importance of Prior Information for Optimization (2013, background): “…patch, y p (for patch p ) can take one of the 11 possible values, 1 for rejection and the rest is for the Pentomino shape classes, illustrated in Figure 2: A similar task has been studied by Fleuret et al. (2011) (at SI appendix Problem 17), who compared the performance of humans vs computers.”
Measuring abstract reasoning in neural networks (2018, background): “Nonetheless, we contend that visual intelligence tests can help to better understand learning and reasoning in machines (Fleuret et al., 2011), provided they are coupled with a principled treatment of generalisation.”
Learning to Make Analogies by Contrasting Abstract Relational Structure (2019, methodology): “Our ﬁrst experiments involve greyscale visual scenes similar to those previously applied to test both human reasoning ability (Raven, 1983; Geary et al., 2000) and reasoning in machine learning models (Bongard, 1967; Fleuret et al., 2011; Barrett et al., 2018).”
Human vs. Computer in Scene and Object Recognition (2014, unknown): “Several previous studies, under the names of humans in the loop, human debugging, finding weak links (and often using Amazon Mechanical Turk) [27, 26, 36, 28, 34], have used humans to estimate the relative strengths and weaknesses of different algorithm components (e.g., [18, 36]).”
25 Years of CNNs: Can We Compare to Human Abstraction Capabilities? (2016, methodology): “Both also compare very favorably to the method presented by Fleuret et al. [3] which achieves a mean accuracy of 0 .”
Invariant object recognition is a personalized selection of invariant features in humans, not simply explained by hierarchical feed-forward vision models (2017, methodology): “The overlaps were calculated using equation (1).”
Learning to Infer and Execute 3D Shape Programs (2019, background): “This part-based, symmetry-aware decomposition is inspired by human perception (Fleuret et al., 2011).”
How Memory Replay in Sleep Boosts Creative Problem-Solving (2018, background): “Importantly, both forms of abstraction also allow for compression, since storage is more compact if similar concepts are coded together rather than separately [44].”
Human and DNN Classification Performance on Images With Quality Distortions (2019, background): “Fleuret et al. (2011) designed a set of synthetic classification tasks to test human and machine vision performance.”
Not-So-CLEVR: learning same–different relations strains feedforward neural networks (2018, background): “[5] found that human observers are capable of learning rather complicated visual rules and generalizing them to new instances from just a few training examples.”
The Notorious Difficulty of Comparing Human and Machine Perception (2020, methodology): “Previous studies (J. Kim et al., 2018; Stabinger et al., 2016) explored how well deep neural networks can learn visual relations by testing them on the Synthetic Visual Reasoning Test (Fleuret et al., 2011).”
Object Level Visual Reasoning in Videos (2018, background): “Humans have this extraordinary ability of performing visual reasoning on very complicated tasks while it remains unattainable for contemporary computer vision algorithms [37,11].”