Interpretable Intuitive Physics Model

By Tian Ye, X. Wang, James Davidson, Abhinav Kumar Gupta. 2018

58 citations recorded by the source corpus.

Humans have a remarkable ability to use physical commonsense and predict the effect of collisions. But do they understand the underlying factors? Can they predict if the underlying factors have changed? Interestingly, in most cases humans can predict the effects of similar collisions with different conditions such as changes in mass, friction, etc. It is postulated this is primarily because we learn to model physics with meaningful latent variables. This does not imply we can estimate the precise values of these meaningful variables (estimate exact values of mass or friction). Inspired by this observation, we propose an interpretable intuitive physics model where specific dimensions in the bottleneck layers correspond to different physical properties. In order to demonstrate that our system models these underlying physical properties, we train our model on collisions of different shapes (cube, cone, cylinder, spheres etc.) and test on collisions of unseen combinations of shapes. Furthermore, we demonstrate our model generalizes well even when similar scenes are simulated with different underlying properties.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

CLEVRER: CoLlision Events for Video REpresentation and Reasoning (2019, neutral): “…et al., 2016; Battaglia et al., 2016; Chang et al., 2017; Agrawal et al., 2016; Finn et al., 2016; Shao et al., 2014; Fire & Zhu, 2016; Pearl, 2009; Ye et al., 2018), either directly from images (Finn et al., 2016; Ebert et al., 2017; Watters et al., 2017; Lerer et al., 2016; Mottaghi et al.,…”
COPHY: Counterfactual Learning of Physical Dynamics (2019, background): “(Ye et al., 2018) build an interpretable intuitive physical model from visual signals using full supervision on the physical properties of each object.”
Grounding Physical Concepts of Objects and Events Through Dynamic Visual Reasoning (2021, background): “Much research has been studying dynamic modeling for physical scenes (Lerer et al., 2016; Battaglia et al., 2013; Mottaghi et al., 2016; Finn et al., 2016; Shao et al., 2014; Fire & Zhu, 2015; Ye et al., 2018; Li et al., 2019b).”
A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithms (2019, background): “More complex hierarchical models, such as neural networks and decision trees, allow the robot to learn taskspecific features that provide better generalization [235, 334, 117, 174, 321, 110, 230, 353, 431].”
Explainable Machine Learning for Scientific Insights and Discoveries (2019, background): “Additionally, [101] constructs the bottleneck layer in their NN to represent physical parameters to predict the outcome of a collision of objects from videos.”
From Recognition to Cognition: Visual Commonsense Reasoning (2018, unknown): “Commonsense Reasoning Our task unifies work involving reasoning about commonsense phenomena, such as physics [54, 84], social interactions [2, 77, 12, 27], procedure understanding [91, 3] and predicting what might happen next in a video [74, 17, 92, 78, 18, 64, 85].”
RELATE: Physically Plausible Multi-Object Scene Synthesis Using Structured Latent Spaces (2020, background): “Physical information extracted from videos can either be explicit such as estimates of velocity or friction [42] or implicitly represented in the latent space, e.”
Doubly Right Object Recognition: A Why Prompt for Visual Rationales (2022, background): “Other sources of external knowledge include interactions [1, 11, 55], physics [31, 59], etc.”
Dynamic Visual Reasoning by Learning Differentiable Physics Models from Video and Language (2021, background): “For example, PhysNet and its variants [48, 21, 74, 75] leverage global or object-centric deep features to predict the physical motion in video frames.”
PIP: Physical Interaction Prediction via Mental Imagery with Span Selection (2021, background): “Many of these intuitive physics models [9, 10, 11, 12, 13, 14] are inspired by the process of mental simulation from the mental physics engine hypothesis in cognitive science research [15, 16, 17, 2].”
ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulation (2020, background): “Differentiable forward predictors that mimic human-level intuitive physical understanding have emerged as being of importance for enabling deep-learning based approaches to model-based planning and control applications [25, 4, 29, 15, 5, 9, 2, 36, 13, 14, 32, 46].”