Specializing Smaller Language Models towards Multi-Step Reasoning

By Yao Fu, Hao-Chun Peng, Litu Ou, Ashish Sabharwal, Tushar Khot. International Conference on Machine Learning, 2023

363 citations recorded by the source corpus.

The surprising ability of Large Language Models (LLMs) to perform well on complex reasoning with only few-shot chain-of-thought prompts is believed to emerge only in very large-scale models (100+ billion parameters). We show that such abilities can, in fact, be distilled down from GPT-3.5 ($\ge$ 175B) to T5 variants ($\le$ 11B). We propose model specialization, to specialize the model's ability towards a target task. The hypothesis is that large models (commonly viewed as larger than 100B) have strong modeling power, but are spread on a large spectrum of tasks. Small models (commonly viewed as smaller than 10B) have limited model capacity, but if we concentrate their capacity on a specific target task, the model can achieve a decent improved performance. We use multi-step math reasoning as our testbed because it is a very typical emergent ability. We show two important aspects of model abilities: (1). there exists a very complex balance/ tradeoff between language models' multi-dimensional abilities; (2). by paying the price of decreased generic ability, we can clearly lift up the scaling curve of models smaller than 10B towards a specialized multi-step math reasoning ability. We further give comprehensive discussions about important design choices for better generalization, including the tuning data format, the start model checkpoint, and a new model selection method. We hope our practice and discoveries can serve as an important attempt towards specialized smaller models in the new research paradigm set by LLMs.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/fbd49b25bdab98c171af49962a41139c73dacbde

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs (2023, background): “A recent line of work has explored student-teacher frameworks with the goal of distilling reasoning capabilities from a stronger teacher to a weaker student (Magister et al., 2023; Fu et al., 2023; Ho et al., 2023; Saha et al., 2023; Mukherjee et al., 2023).”
Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agent (2023, background): “For instance, Fu et al. (2023) and Magister et al. (2023) have successfully distilled the reasoning ability of LLMs into smaller models.”
Alphazero-like Tree-Search can Guide Large Language Model Decoding and Training (2023, neutral): “…on four tasks with different search widths and depths, including the mathematical reasoning task GSM8k (Cobbe et al., 2021), mathematical planning task Game24 (Yao et al., 2023), logical reasoning task PrOntoQA (Saparov & He, 2022), and RLHF alignment task using synthetic RLHF data (Dahoas).”
LLM-Assisted Code Cleaning For Training Accurate Code Generators (2023, background): “We next study improving planning in a supervised learning setup similar to prior works (Fu et al., 2023; Li et al., 2023b).”
MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models (2023, background): “However, for solving challenging mathematical problems ( e.g. , GSM8K [13]), there is still a large performance gap [19, 25, 45].”
A Survey of Large Language Models (2023, neutral): “For example, the researchers in [265] specialize the ability of mathematical reasoning”
Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes (2023, methodology): “Knowledge distillation has been successfully used to transfer knowledge from larger, more competent teacher models into smaller student models affordable for practical applications (Buciluˇa et al., 2006; Hinton et al., 2015; Beyer et al., 2022; West et al., 2021; Fu et al., 2023).”
Recommendation as Instruction Following: A Large Language Model Empowered Recommendation Approach (2023, positive): “In this new recommendation paradigm, a user can freely express the specific needs via instruction texts, while the recommender system will follow the user’s instructions for making accurate recommendations based on powerful LLMs.”
ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving (2023, methodology): “Tool-Augmented Language Models Augmenting LLMs with tools can largely alleviate LLMs’ limitations and improve reasoning and generation performance (Parisi et al., 2022; Mialon et al., 2023; Yao et al., 2023).”