Teaching Autoregressive Language Models Complex Tasks By Demonstration

By Gabriel Recchia. 2021

23 citations recorded by the source corpus.

This paper demonstrates that by fine-tuning an autoregressive language model (GPT-Neo) on appropriately structured step-by-step demonstrations, it is possible to teach it to execute a mathematical task that has previously proved difficult for Transformers - longhand modulo operations - with a relatively small number of examples. Specifically, we fine-tune GPT-Neo to solve the numbers__div_remainder task from the DeepMind Mathematics Dataset; Saxton et al. (arXiv:1904.01557) reported below 40% accuracy on this task with 2 million training examples. We show that after fine-tuning on 200 appropriately structured demonstrations of solving long division problems and reporting the remainders, the smallest available GPT-Neo model achieves over 80% accuracy. This is achieved by constructing an appropriate dataset for fine-tuning, with no changes to the learning algorithm. These results suggest that fine-tuning autoregressive language models on small sets of well-crafted demonstrations may be a useful paradigm for enabling individuals without training in machine learning to coax such models to perform some kinds of complex multi-step tasks.

Subfields: Artificial Intelligence, Large Language Models, Prompting & Reasoning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Andrew Kyle Lampinen
2. Felix Hill
3. Adam Santoro
4. Yoshua Bengio
5. Sergey Levine
6. James L. McClelland
7. Timothy P. Lillicrap
8. Stephanie C. Y. Chan

In Large Language Models

1. Felix Hill
2. Yoav Levine
3. James L. McClelland
4. Xifeng Yan
5. Kory Wallace Mathewson
6. Zekun Li
7. Timothy P. Lillicrap
8. Amnon Shashua

In Prompting & Reasoning

1. Kory Wallace Mathewson
2. Felix Hill
3. Yoav Levine
4. Xifeng Yan
5. Zekun Li
6. Amnon Shashua
7. Andrew Kyle Lampinen
8. Antonia Creswell

Citations in the corpus

• Chain of Thought Prompting Elicits Reasoning in Large Language Models (2022, neutral): “Perhaps the most-related work here is Recchia (2021), which shows that finetuning enables longhand module operations, which has previously been difficult for performers.”
• Can language models learn from explanations in context? (2022, background): “, 2022), especially when they are augmented with external scratchpads to store intermediate computations (Nye et al., 2021; Recchia, 2021).”
• Symbolic Behaviour in Artificial Intelligence (2021, background): “Similarly, while it is difficult for transformers to learn to perform division from problems and answers alone, even with millions of examples [84], a pretrained language model endowed with a virtual notepad and trained on only 200 detailed demonstrations of long division can achieve 80% generalizat”
• Limitations of Language Models in Arithmetic and Symbolic Induction (2022, methodology): “A similar method with more detailed demonstration is introduced in (Recchia, 2021).”
• Sub-Task Decomposition Enables Learning in Sequence to Sequence Tasks (2022, methodology): “This approach has recently been widely adopted, e.g. , by Rajani et al. (2019); Cobbe et al. (2021); Pi˛ekos et al. (2021); Recchia (2021); Nye et al. (2022); Wei et al. (2022); Zelikman et al. (2022).”