The Unreliability of Explanations in Few-Shot In-Context Learning

By Xi Ye, Greg Durrett. 2022

37 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Large Language Models, Prompting & Reasoning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Large Language Models

In Prompting & Reasoning

Citations in the corpus

Chain of Thought Prompting Elicits Reasoning in Large Language Models (2022, background): “Overall, there are no guarantees that the reasoning processes generated by large language models are coherent or factually correct, as underscored by the recent work evaluating the factuality of language model generations and explanations (Maynez et al., 2020; Rashkin et al., 2021; Ye and Durrett, 2”
Self-Consistency Improves Chain of Thought Reasoning in Language Models (2022, background): “In additional experiments, we show self-consistency can robustly boost performance on NLP tasks where adding a chain-of-thought might hurt performance compared to standard prompting (Ye & Durrett, 2022).”
Large Language Models Can Self-Improve (2022, neutral): “Majority Voting by answer ···”
Rationale-Augmented Ensembles in Language Models (2022, background): “…useful for supervised learning in natural language tasks when added in the training data (Zaidan et al., 2007; Ling et al., 2017; Cobbe et al., 2021; Zelikman et al., 2022), but it remains unclear whether such rationales can be reliably useful in few-shot in-context learning (Ye & Durrett, 2022).”
Measuring and Narrowing the Compositionality Gap in Language Models (2022, background): “Ye and Durrett (2022) use chain of thought to answer compositional questions that are presented with a context paragraph.”
Self-Polish: Enhance Reasoning in Large Language Models via Problem Refinement (2023, background): “When confronted with real-world reasoning problems, which may be excessively long, ambiguous, unclear in focus, or contain irrelevant information (Zellers et al., 2018; Shi et al., 2023), LLMs often exhibit poor reasoning performance (Ye and Durrett, 2022; Shi et al., 2023).”
Are Hard Examples also Harder to Explain? A Study with Human and Model-Generated Explanations (2022, background): “A number of concurrent works have also explored free-text explanations for incontext learning in various reasoning tasks (Nye et al., 2021; Chowdhery et al., 2022; Wei et al., 2022; Lampinen et al., 2022; Wang et al., 2022; Ye and Durrett, 2022), primarily focusing on improving model performance wit”
Mathematical Capabilities of ChatGPT (2023, background): “…was already considerable; adding such prompt engineering methods would have further raised the evaluation cost while also increasing the complexity of our non-trivial evaluation methodology and 3) for GPT-3 and certain datasets and prompting engineering techniques, performance was degraded [63].”
Honest Students from Untrusted Teachers: Learning an Interpretable Question-Answering Pipeline from a Pretrained Language Model (2022, methodology): “We use a very similar NLI system, trained 1 The link between explanations and calibration was explored by Ye and Durrett (2022), who work in a chain-of-thought prompting framework.”
Reasoning with Language Model Prompting: A Survey (2022, background): “Calibrator (Ye and Durrett, 2022) Manual Explanations InstructGPT few-shot fine-tune Self-Consistency (Wang et al.”
Rethinking with Retrieval: Faithful Large Language Model Inference (2022, background): “, 2021), calibrating model predictions based on the reliability of the explanations (Ye and Durrett, 2022), and promoting selfconsistency over diverse reasoning paths (Wang et al.”
Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanations (2022, background): “Despite their success, recent observations reveal that LM-generated explanations are unreliable, as they often lack logical consistency and are not factually grounded (Ye and Durrett, 2022; Kassner and Schütze, 2020).”
On Second Thought, Let’s Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning (2022, methodology): “Furthermore, reasoning strategies used by LLMs are opaque: models are prone to generating unreliable explanations (Ye and Durrett, 2022) and may not understand provided incontext examples/demonstrations at all (Min et al.”