Rethinking with Retrieval: Faithful Large Language Model Inference

By Hangfeng He, Hongming Zhang, Dan Roth. 2022

217 citations recorded by the source corpus.

Despite the success of large language models (LLMs) in various natural language processing (NLP) tasks, the stored knowledge in these models may inevitably be incomplete, out-of-date, or incorrect. This motivates the need to utilize external knowledge to assist LLMs. Unfortunately, current methods for incorporating external knowledge often require additional training or fine-tuning, which can be costly and may not be feasible for LLMs. To address this issue, we propose a novel post-processing approach, rethinking with retrieval (RR), which retrieves relevant external knowledge based on the decomposed reasoning steps obtained from the chain-of-thought (CoT) prompting. This lightweight approach does not require additional training or fine-tuning and is not limited by the input length of LLMs. We evaluate the effectiveness of RR through extensive experiments with GPT-3 on three complex reasoning tasks: commonsense reasoning, temporal reasoning, and tabular reasoning. Our results show that RR can produce more faithful explanations and improve the performance of LLMs.

Subfields: Artificial Intelligence, Retrieval-Augmented Generation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Mohit Bansal
  2. Jianfeng Gao
  3. Yejin Choi
  4. Luke S. Zettlemoyer
  5. Percy Liang
  6. Graham Neubig
  7. Hannaneh Hajishirzi
  8. Noah A. Smith

In Retrieval-Augmented Generation

  1. Danqi Chen
  2. Jianfeng Gao
  3. Edouard Grave
  4. Weizhu Chen
  5. Daniel Khashabi
  6. Pengcheng He
  7. Tianyu Gao
  8. Jiatong Yu

Citations in the corpus