Dataless Knowledge Fusion by Merging Weights of Language Models

By Xisen Jin, Xiang Ren, Daniel Preotiuc-Pietro, Pengxiang Cheng. International Conference on Learning Representations, 2022

406 citations recorded by the source corpus.

Fine-tuning pre-trained language models has become the prevalent paradigm for building downstream NLP models. Oftentimes fine-tuned models are readily available but their training data is not, due to data privacy or intellectual property concerns. This creates a barrier to fusing knowledge across individual models to yield a better single model. In this paper, we study the problem of merging individual models built on different training data sets to obtain a single model that performs well both across all data set domains and can generalize on out-of-domain data. We propose a dataless knowledge fusion method that merges models in their parameter space, guided by weights that minimize prediction differences between the merged model and the individual models. Over a battery of evaluation settings, we show that the proposed method significantly outperforms baselines such as Fisher-weighted averaging or model ensembling. Further, we find that our method is a promising alternative to multi-task learning that can preserve or sometimes improve over the individual models without access to the training data. Finally, model merging is more efficient than training a multi-task model, thus making it applicable to a wider set of scenarios.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/3ff08b5ca57e786d8af7b204ef94c9972bd9a61e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Mohit Bansal
2. Prateek Yadav
3. Derek Tam
4. Colin Raffel
5. Fei Huang
6. Leshem Choshen
7. Yu Bowen
8. Yongbin Li

Citations in the corpus

• BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domains (2024, background): “Recent studies suggest merging pre-trained models to enhance performance and out-of-domain generalization (Cha et al., 2021; Arpit et al., 2022; Wortsman et al., 2022; Jin et al., 2023; Ilharco et al., 2023).”
• Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch (2023, methodology): “WizardCoder-Python-13B is not selected as it is fine-tuned from CodeLlama-13b-Python.”
• TIES-Merging: Resolving Interference When Merging Models (2023, methodology): “This interference between influential parameters might explain why the performance gap between the merged model and multitask-trained model increases as the number of models increases [31].”
• LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Composition (2023, neutral): “Prior works (Matena & Raffel, 2021; Jin et al., 2023; Yadav et al., 2023; Wu et al., 2023a) .”