Fine-tuned Language Models are Continual Learners

By Thomas Scialom, Tuhin Chakrabarty, Smaranda Muresan. 2022

185 citations recorded by the source corpus.

Recent work on large language models relies on the intuition that most natural language processing tasks can be described via natural language instructions and that models trained on these instructions show strong zero-shot performance on several standard datasets. However, these models even though impressive still perform poorly on a wide range of tasks outside of their respective training and evaluation sets.To address this limitation, we argue that a model should be able to keep extending its knowledge and abilities, without forgetting previous skills. In spite of the limited success of Continual Learning, we show that Fine-tuned Language Models can be continual learners.We empirically investigate the reason for this success and conclude that Continual Learning emerges from self-supervision pre-training. Our resulting model Continual-T0 (CT0) is able to learn 8 new diverse language generation tasks, while still maintaining good performance on previous tasks, spanning in total of 70 datasets. Finally, we show that CT0 is able to combine instructions in ways it was never trained for, demonstrating some level of instruction compositionality.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Large Language Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Large Language Models

Citations in the corpus

Self-Instruct: Aligning Language Models with Self-Generated Instructions (2022, background): “A series of works have found evidence that vanilla language models can be effective at following general language instructions if tuned with annotated “instructional” data – datasets containing language instructional commands and their desired outcome based on human judgement (Weller et al., 2020; M”
The Role of ChatGPT in Data Science: How AI-Assisted Conversational Interfaces Are Revolutionizing the Field (2023, background): “It has been shown that ﬁne-tuned language models can be continual learners, thereby giving us an indication of the future capabilities of innovations such as ChatGPT [21].”
A Comprehensive Overview of Large Language Models (2023, unknown): “Concatenating fine-tuning data with a few randomly selected pre-training samples in every iteration avoids network forgetting [162], [132].”
LoRA Learns Less and Forgets Less (2024, background): “A common approach to mitigate forgetting involves “replaying” source-domain data during continual learning, which can be done by storing the data in a memory buffer, or generating it on the fly (Lesort et al., 2022; Scialom et al., 2022; Sun et al., 2019).”
Style Over Substance: Evaluation Biases for Large Language Models (2023, unknown): “Therefore, we recommend a multi-dimensional evaluation approach for assessing machine-generated text rather than relying on a single metric, and advise caution when using crowd annotators for LLM evaluations.”
An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-Tuning (2023, unknown): “Then we analyze the factors that affect the forgetting extent, such as model scales, model architectures, and general instruction tuning in Section V-B–V-E, respectively.”