LAMBADA: Backward Chaining for Automated Reasoning in Natural Language

By Seyed Mehran Kazemi, Najoung Kim, Deepti Bhatia, Xinyuan Xu, Deepak Ramachandran. 2022

105 citations recorded by the source corpus.

Remarkable progress has been made on automated reasoning with natural text, by using Large Language Models (LLMs) and methods such as Chain-of-Thought prompting and Selection-Inference. These techniques search for proofs in the forward direction from axioms to the conclusion, which suffers from a combinatorial explosion of the search space, and thus high failure rates for problems requiring longer chains of reasoning. The classical automated reasoning literature has shown that reasoning in the backward direction (i.e. from intended conclusion to supporting axioms) is significantly more efficient at proof-finding. Importing this intuition into the LM setting, we develop a Backward Chaining algorithm, called LAMBADA, that decomposes reasoning into four sub-modules, that are simply implemented by few-shot prompted LLM inference. We show that LAMBADA achieves sizable accuracy boosts over state-of-the-art forward reasoning methods on two challenging logical reasoning datasets, particularly when deep and accurate proof chains are required.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Augmented Language Models: a Survey (2023, methodology): “Finally, Kazemi et al. (2022) decompose a given problem in a backward fashion: starting from the goal and a set of rules, the system decompose the goal into sub-goals and recursively check whether the sub-goals can be proved.”
Reasoning with Language Model Prompting: A Survey (2022, unknown): “, 2022), LAMBADA (Kazemi et al., 2022), Iter-Decomp (Reppert et al.”
Towards Reasoning in Large Language Models: A Survey (2022, methodology): “Kazemi et al. (2022) suggest using backward chaining, i.e., from goal to the set of facts that support it, instead of forward chaining like Creswell et al. (2022); Creswell and Shanahan (2022).”
The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity (2025, unknown): “For instance, generating a Chain of Thought (CoT) [15, 16, 17, 18] and incorporating self-verification [19, 20, 21] prior to the final answer have been shown to improve model performance.”