Distilling Reasoning Capabilities into Smaller Language Models

By Kumar Shridhar, Alessandro Stolfo, Mrinmaya Sachan. 2022

270 citations recorded by the source corpus.

Step-by-step reasoning approaches like chain of thought (CoT) have proved to be very effective in inducing reasoning capabilities in large language models. However, the success of the CoT approach is fundamentally tied to the model size, and billion parameter-scale models are often needed to get CoT to work. In this paper, we propose a knowledge distillation approach that leverages the step-by-step CoT reasoning capabilities of larger models and distills these abilities into smaller models. In this work, we propose an alternative reasoning scheme, Socratic CoT, that learns a decomposition of the original problem into a sequence of subproblems and uses it to guide the intermediate reasoning steps. We use Socratic CoT to train a combination of two small distilled models: a problem decomposer and a subproblem solver. In practice, given a new problem, the two distilled models work in sync to decompose and solve complex problems. On multiple reasoning datasets (GSM8K, StrategyQA, and SVAMP), our proposed distillation strategies boosts the performance of smaller models over 70% compared to the baselines. Finally, we investigate when Socratic CoT is an effective alternative to CoT, demonstrating cases where a much smaller model (GPT-2 large) can outperform a 10X larger model (GPT-3 6B). Our code is available here: https://github.com/kumar-shridhar/Distiiling-LM

Subfields: Artificial Intelligence, Large Language Models, Prompting & Reasoning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Large Language Models

In Prompting & Reasoning

Citations in the corpus

Large Language Models Can Self-Improve (2022, positive): “Recent works (Magister et al., 2022; dhar et al., 2023; Ho et al., 2023) demonstrated that distillation on explanations generated from large models can increase the reasoning abilities of smaller models with ground truth filtering.”
Large Language Models as Commonsense Knowledge for Large-Scale Task Planning (2023, methodology): “There are also various ways to enhance runtime performance like using smaller LLMs like Llama [35, 36] or distilling LLM’s knowledge into a smaller model [30, 16, 23].”
MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models (2023, background): “Knowledge Distillation [21, 24] transfers knowledge from a larger teacher model to a smaller student model, achieving promising performance in many applications [22, 48, 56, 63], Recently, [19, 25– 27, 37, 45, 64] propose to transfer reasoning abilities from LLMs ( e.g. , GPT-3.”