Teaching Pre-Trained Models to Systematically Reason Over Implicit Knowledge

By Alon Talmor, Oyvind Tafjord, Peter Clark, Yoav Goldberg, Jonathan Berant. 2020

38 citations recorded by the source corpus.

To what extent can a neural network systematically reason over symbolic facts? Evidence suggests that large pre-trained language models (LMs) acquire some reasoning capacity, but this ability is difficult to control. Recently, it has been shown that Transformer-based models succeed in consistent reasoning over explicit symbolic facts, under a "closed-world" assumption. However, in an open-domain setup, it is desirable to tap into the vast reservoir of implicit knowledge already encoded in the parameters of pre-trained LMs. In this work, we provide a first demonstration that LMs can be trained to reliably perform systematic reasoning combining both implicit, pre-trained knowledge and explicit natural language statements. To do this, we describe a procedure for automatically generating datasets that teach a model new reasoning skills, and demonstrate that models learn to effectively perform inference which involves implicit taxonomic and world knowledge, chaining and counting. Finally, we show that "teaching" models to reason generalizes beyond the training distribution: they successfully compose the usage of multiple reasoning skills in single examples. Our work paves a path towards open-domain systems that constantly improve by interacting with users who can instantly correct a model by adding simple natural language statements.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Large Language Models, Prompting & Reasoning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Large Language Models

In Prompting & Reasoning

Citations in the corpus

RuleBERT: Teaching Soft Rules to Pre-Trained Language Models (2021, background): “PLMs have been shown to have some reasoning capabilities (Talmor et al., 2020b), but fail on basic reasoning tasks (Talmor et al., 2020a) and are inconsistent (Elazar et al., 2021), especially when it comes to negation (Kassner and Schütze, 2020).”
CommonsenseQA 2.0: Exposing the Limits of AI through Gamification (2021, neutral): “Our work joins a line of previously introduced benchmarks targeting commonsense reasoning 365 [12, 13, 29, 15] as well as probing for other reasoning abilities [32, 33, 34, 35, 36, 37, 38].”
When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Data (2021, methodology): “Talmor et al. (2020) systematically study RoBERTa’s ability to combine pieces of knowledge in a reasoning task by including relevant factoids in the text input.”
Datasheet for the Pile (2022, methodology): “Papers that train models on datasets that include the OpenSubstitles subset of the Pile include Luo et al. [2021], Askell et al. [2021] DM Mathematics: DMMathematics has been used extensively, including by [Cho et al., 2019, Qi and Wu, 2019, Talmor et al., 2020, Dinu et al., 2020, Firestone, 2020].”
Learning to repair: Repairing model output errors after deployment using a dynamic memory of feedback (2021, unknown): “(4) Provide additional information: In TeachY-ourAI (Talmor et al., 2020), given a wrong answer to a question, users can enter NL facts and rules to use as context when reasking the question, to (ideally) produce the correct answer.”
Shepherd Pre-trained Language Models to Develop a Train of Thought: An Iterative Prompting Approach (2022, methodology): “The abbreviation here comes from the phrase “Leap-ofThought” in the paper title of (Talmor et al., 2020b).”
Interscript: A dataset for interactive learning of scripts through error feedback (2021, methodology): “Finally, the model being taught is treated either as (i) a black box (as in machine teaching (Dasgupta et al. 2019; Talmor et al. 2020)) or (ii) the beliefs of the model are in some form exposed to feedback (as in interactive semantic parsing (Elgohary et al. 2021)).”