ESPRIT: Explaining Solutions to Physical Reasoning Tasks

By Nazneen Rajani, Rui Zhang, Y. Tan, Stephan Zheng, Jeremy C. Weiss, Aadit Vyas, Abhijit Gupta, Caiming Xiong, Richard Socher, Dragomir R. Radev. 2020

29 citations recorded by the source corpus.

Neural networks lack the ability to reason about qualitative physics and so cannot generalize to scenarios and tasks unseen during training. We propose ESPRIT, a framework for commonsense reasoning about qualitative physics in natural language that generates interpretable descriptions of physical events. We use a two-step approach of first identifying the pivotal physical events in an environment and then generating natural language descriptions of those events using a data-to-text approach. Our framework learns to generate explanations of how the physical simulation will causally evolve so that an agent or a human can easily reason about a solution using those interpretable descriptions. Human evaluations indicate that ESPRIT produces crucial fine-grained details and has high coverage of physical concepts compared to even human annotations. Dataset, code and documentation are available at https://github.com/salesforce/esprit.

Subfields: Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Safety, Robustness & Interpretability

Citations in the corpus

DART: Open-Domain Structured Data Record to Text Generation (2020, background): “, 2015, 2016), and commonsense reasoning (Lin et al., 2019; Rajani et al., 2020).”
Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processing (2021, background): “[101] Nazneen Fatema Rajani, Rui Zhang, Yi Chern Tan, Stephan Zheng, Jeremy Weiss, Aadit Vyas, Abhijit Gupta, Caiming Xiong, Richard Socher, and Dragomir Radev.”
Right for the Right Concept: Revising Neuro-Symbolic Concepts by Interacting with their Explanations (2020, background): “research/NeSyXIL Several studies have investigated methods that produce explanations other than these visual explanations, such as multi-modal explanations [36, 54, 40], including visual and logic rule explanations [1, 39].”
Teach Me to Explain: A Review of Datasets for Explainable NLP (2021, background): “…28,180 (1 or 3) (Zellers et al., 2019a) visual commonsense reasoning crowd ∼ 290K E-SNLI-VE †† (Do et al., 2020) visual-textual entailment crowd 11,335 (3) ‡ ESPRIT †† (Rajani et al., 2020) reasoning about qualitative physics crowd 2441 (2) VLEP †† (Lei et al., 2020) future event prediction…”