Faithful Chain-of-Thought Reasoning

By Qing Lyu, Shreya Havaldar, Adam Stein, Li Zhang, Delip Rao, Eric Wong, Marianna Apidianaki, Chris Callison-Burch. International Joint Conference on Natural Language Processing, 2023

427 citations recorded by the source corpus.

While Chain-of-Thought (CoT) prompting boosts Language Models' (LM) performance on a gamut of complex reasoning tasks, the generated reasoning chain does not necessarily reflect how the model arrives at the answer (aka. faithfulness). We propose Faithful CoT, a reasoning framework involving two stages: Translation (Natural Language query $\rightarrow$ symbolic reasoning chain) and Problem Solving (reasoning chain $\rightarrow$ answer), using an LM and a deterministic solver respectively. This guarantees that the reasoning chain provides a faithful explanation of the final answer. Aside from interpretability, Faithful CoT also improves empirical performance: it outperforms standard CoT on 9 of 10 benchmarks from 4 diverse domains, with a relative accuracy gain of 6.3% on Math Word Problems (MWP), 3.4% on Planning, 5.5% on Multi-hop Question Answering (QA), and 21.4% on Relational Inference. Furthermore, with GPT-4 and Codex, it sets the new state-of-the-art few-shot performance on 7 datasets (with 95.0+ accuracy on 6 of them), showing a strong synergy between faithfulness and accuracy.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/b115c1e1e9e51f8ad7d47b745bc04e29a654b84d

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Reasoning with Language Model is Planning with World Model (2023, background): “Moreover, based on LLMs’ powerful programming ability [37, 29, 36], some recent works first translate natural language instructions into the executable programming languages, such as Planning Domain Description Language (PDDL), and runs classical planning algorithms, such as LLM+P [36].”
Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning (2023, background): “Recent studies (Golovneva et al., 2023; Ribeiro et al., 2023b; Lyu et al., 2023) found that LLMs occasionally make unfaithful reasoning, i.e. , the derived conclusion does not follow the previously generated reasoning chain.”
Measuring Faithfulness in Chain-of-Thought Reasoning (2023, methodology): “Lyu et al. (2023) generate a program in a domain-speciﬁc language and execute the program (e.g., using a Python interpreter) to produce the ﬁnal answer; this process ensures that the generating program is not post-hoc but rather directly used to produce the ﬁnal answer.”
Leveraging Pre-trained Large Language Models to Construct and Utilize World Models for Model-based Task Planning (2023, methodology): “Besides, the goal specification can be obtained by using an LLM to parse the user’s command and convert it into a symbolic form, as done previously in [30, 58, 32].”
Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces (2024, positive): “Self-explanations are a prevailing approach on par with traditional model explanations like LIME saliency maps [71] for understanding LLM-generated responses [25, 32, 53] and are widely used in analyzing language model behavior [66, 99].”
Reasoning Models Don't Always Say What They Think (2025, positive): “To improve faithfulness, researchers have developed test-time methods using question decomposition (Radhakrishnan et al., 2023), symbolic reasoning (Lyu et al., 2023), and causal mediation analysis on counterfactuals (Paul et al., 2024 (Clymer et al., 2024).”
The rise and potential of large language model based agents: a survey (2025, positive): “In contrast, other studies like the CoT-series employ an adaptive strategy, where they plan and address sub-tasks one at a time, allowing for more ﬂuidity in handling intricate tasks in their entirety [76, 77, 166].”
A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions (2023, neutral): “…tasks through Chain-of-Thought prompting (Wei et al., 2022), they occasionally exhibit unfaithful reasoning characterized by inconsistencies within the reasoning steps or conclusions that do not logically follow the reasoning chain (Golovneva et al., 2022; Ribeiro et al., 2023; Lyu et al., 2023).”
A Survey of Large Language Models (2023, positive): “Concretely, LLMs may generate the correct answer following an invalid reasoning path, or produce a wrong answer after a correct reasoning process [33, 374], leading to inconsistency between the derived answer and the rea-”