A Comparative Evaluation of Approximate Probabilistic Simulation and Deep Neural Networks as Accounts of Human Physical Scene Understanding

By Renqiao Zhang, Jiajun Wu, Chengkai Zhang, William T. Freeman, Josh Tenenbaum. 2016

53 citations recorded by the source corpus.

Humans demonstrate remarkable abilities to predict physical events in complex scenes. Two classes of models for physical scene understanding have recently been proposed: "Intuitive Physics Engines", or IPEs, which posit that people make predictions by running approximate probabilistic simulations in causal mental models similar in nature to video-game physics engines, and memory-based models, which make judgments based on analogies to stored experiences of previously encountered scenes and physical outcomes. Versions of the latter have recently been instantiated in convolutional neural network (CNN) architectures. Here we report four experiments that, to our knowledge, are the first rigorous comparisons of simulation-based and CNN-based models, where both approaches are concretely instantiated in algorithms that can run on raw image inputs and produce as outputs physical judgments such as whether a stack of blocks will fall. Both approaches can achieve super-human accuracy levels and can quantitatively predict human judgments to a similar degree, but only the simulation-based models generalize to novel situations in ways that people do, and are qualitatively consistent with systematic perceptual illusions and judgment asymmetries that people show.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Cognitive Science & Psychology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Cognitive Science & Psychology

Citations in the corpus

Learning Physical Intuition of Block Towers by Example (2016, neutral): “These models are purely bottom-up in nature, in contrast to existing approaches which rely on complex top-down graphics engines.”
Physics 101: Learning Physical Object Properties from Unlabeled Videos (2016, background): “Recently, researchers started to tackle concrete scenarios for understanding physics from vision [11, 15, 18, 19, 23, 24, 25, 27], some building on recent advances in deep learning.”
IntPhys: A Framework and Benchmark for Visual Intuitive Physics Reasoning (2018, methodology): “Recent studies have investigated models for predicting the stability and forward modeling the dynamics of towers of blocks ([8, 35, 74, 39, 46, 40]).”
ESPRIT: Explaining Solutions to Physical Reasoning Tasks (2020, unknown): “Earlier works explored using probabilistic simulations to train physical inference through physical simulations (Battaglia et al., 2013; Zhang et al., 2016).”
The Notorious Difficulty of Comparing Human and Machine Perception (2020, background): “In the attempt to probe intuition in machine models, DNNs are compared to intuitive physics engines, i.e. probabilistic models that simulate physical events (R. Zhang, Wu, Zhang, Freeman, & Tenenbaum, 2016).”
Mind Games: Game Engines as an Architecture for Intuitive Physics. (2017, background): “…useful for limited AI settings, but it belies that fact that the network does not generalize well even to quite similar scenes (for example, in judging towers composed of more blocks than the training set), nor does it display asymmetries shown by both humans and physics-engine based models [59].”
Interpretable Intuitive Physics Model (2018, background): “Physical reasoning and learning physical commonsense has raised a lot of interest in recent years [29,17,16,28,18,1,31,5].”
PHYRE: A New Benchmark for Physical Reasoning (2019, methodology): “Early computational instantiations of this framework used probabilistic models over physical simulations [3, 61], while more recent methods use feedforward neural networks trained to make pixelwise [11, 24, 34, 60] or qualitative [12, 26, 27, 34, 61] predictions about the future, sometimes in conjuc”