Leap-Of-Thought: Teaching Pre-Trained Models to Systematically Reason Over Implicit Knowledge

By Alon Talmor, Oyvind Tafjord, Peter Clark, Yoav Goldberg, Jonathan Berant. 2020

118 citations recorded by the source corpus.

Subfields: Large Language Models, Prompting & Reasoning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Large Language Models

In Prompting & Reasoning

Citations in the corpus

Chain of Thought Prompting Elicits Reasoning in Large Language Models (2022, background): “, 2019, inter alia) or generates synthetic datasets (Talmor et al., 2020; Zelikman et al., 2022).”
Scaling Instruction-Finetuned Language Models (2022, background): “, 2022, inter alia) or focuses on models of substantially smaller scale (Ling et al., 2017; Camburu et al., 2018; Rajani et al., 2019; Talmor et al., 2020; Zelikman et al., 2022).”
Memory-assisted prompt editing to improve GPT-3 after deployment (2022, background): “Building upon the recent success of injecting commonsense in the input (Lewis et al., 2020; Talmor et al., 2020), we pro-pose a novel approach of injecting knowledge in the input via interactive feedback from an end-user.”
Flexible Generation of Natural Language Deductions (2021, background): “Systems like PRover (Saha et al., 2020) and Leap-of-Thought (Talmor et al., 2020) have some broadly similar goals to ours, but only retrieve facts and do not generate novel conclusions.”
Do Language Models Have Beliefs? Methods for Detecting, Updating, and Visualizing Model Beliefs (2021, methodology): “Here, we adapt data from Talmor et al. (2020) to measure consistency under entailment (including for contrapositives), and we use the Wikidata5m dataset (Wang et al., 2021b) to construct logically neutral belief pairs for checking that models do treat the beliefs as independent.”
Thinking Aloud: Dynamic Context Generation Improves Zero-Shot Reasoning Performance of GPT-2 (2021, methodology): “Moreover, Kassner and Schütze [2020], extending the LAMA probe by Petroni et al. [2020b], show that LMs are vulnerable to mispriming effects and have major difﬁculties in getting negations right [consistent with Talmor et al., 2020a].”
Natural Language Deduction through Search over Statement Compositions (2022, background): “Finally, the Leap of Thought system (Talmor et al., 2020) tackles some related ideas in a discriminative reasoning framework.”
BeliefBank: Adding Memory to a Pre-Trained Language Model for a Systematic Notion of Belief (2021, background): “Such mechanisms may be symbolic or neural in nature, e.g., (Talmor et al., 2020).”
Making Retrieval-Augmented Language Models Robust to Irrelevant Context (2023, background): “This can be partially explained by the ability of models to combine retrieval and their parametric knowledge (Talmor et al., 2020; Zhong et al., 2023; Cohen et al., 2023).”
Towards Reasoning in Large Language Models: A Survey (2022, background): “Talmor et al. (2020) train RoBERTa (Liu et al., 2019) There are two major limitations of fully supervised finetuning.”
On the Paradox of Learning to Reason from Data (2022, background): “With these evidences (and with even fewer evidences), it has been claimed that neural models can learn to reliably emulate the correct reasoning function (Clark et al., 2020; Talmor et al., 2020).”
Iteratively Prompt Pre-trained Language Models for Chain of Thought (2022, methodology): “…multi-step reasoning which include annotations for knowledge statements relevant to the queries: 2WikiMultiHopQA (abbreviated as 2Wiki) (Ho et al., 2020), R4C (Inoue et al., 2020), and a scien-tiﬁc commonsense reasoning dataset (abbreviated as LoT 2 ) constructed by (Talmor et al., 2020b).”
Reasoning with Transformer-based Models: Deep Learning, but Shallow Reasoning (2021, methodology): “Transformer-based models are able to perform such kind of reasoning [Clark et al., 2020, Talmor et al., 2020, Betz et al., 2020] without any external knowledge, if both the rules and the facts are mentioned explicitly in the text.”
Representing Numbers in NLP: a Survey and a Vision (2021, unknown): “Counting tasks have been proposed in computer vision (Testolin et al., 2020) as well as in NLP (Postma et al., 2018; Talmor et al., 2020).”
A Review on Language Models as Knowledge Bases (2022, methodology): “Hase et al. (2021) measure consistency under entailment, including contrapositives, after updating beliefs of a LM. Specifically, they adapt the LeapOfThought dataset (Talmor et al., 2020b) such that each datapoint has a main fact (e.g.”
Rethinking with Retrieval: Faithful Large Language Model Inference (2022, background): “Explicit rules have also been added to inputs to improve reasoning ability over implicit knowledge (Talmor et al., 2020).”
Relational World Knowledge Representation in Contextual Language Models: A Review (2021, methodology): “The cloze prompting protocol (Taylor, 1953 and Figure 2) is a direct approach for extracting and evaluating KB-like knowledge in pretrained LMs.”