LMFlow: An Extensible Toolkit for Finetuning and Inference of Large Foundation Models

By Shizhe Diao, Hanze Dong, Kashun Shum, Jipeng Zhang, Wei Xiong, T. Zhang. 2023

82 citations recorded by the source corpus.

Foundation models have demonstrated a great ability to achieve general human-level intelligence far beyond traditional approaches. As the technique keeps attracting attention from the AI community, more and more foundation models have become publicly available.However, most of those models exhibit a major deficiency in specialized-domain and specialized-task applications, where the step of domain- and task-aware finetuning is still required to obtain scientific language models. As the number of available foundation models and specialized tasks keeps growing, the job of training scientific language models becomes highly nontrivial. In this paper, we take the first step to address this issue. We introduce an extensible and lightweight toolkit, LMFlow, which aims to simplify the domain- and task-aware finetuning of general foundation models.LMFlow offers a complete finetuning workflow for a foundation model to support specialized training with limited computing resources.Furthermore, it supports continuous pretraining, instruction tuning, parameter-efficient finetuning, alignment tuning, inference acceleration, long context generalization, model customization, and even multimodal finetuning, along with carefully designed and extensible APIs. This toolkit has been thoroughly tested and is available at https://github.com/OptimalScale/LMFlow.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Electrical and Electronic Engineering, Large Language Models, ML Systems & Hardware

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Ee-Peng Lim
2. Yihuai Lan
3. Wei Xiong
4. Hanze Dong
5. Lei Wang
6. Jindong Wang
7. Zhengran Zeng
8. Zhiqiang Hu

In Computer Vision and Pattern Recognition

1. Lei Wang
2. Jindong Wang
3. Zhiqiang Hu
4. Ee-Peng Lim
5. Soujanya Poria
6. Lidong Bing
7. Hao Chen
8. Yidong Wang

In Electrical and Electronic Engineering

1. Soujanya Poria
2. Doyen Sahoo

In Large Language Models

1. Lei Wang
2. Soujanya Poria
3. Lidong Bing
4. Wei Xiong
5. Hanze Dong
6. Cunxiang Wang
7. Doyen Sahoo

Citations in the corpus

• Judging LLM-as-a-judge with MT-Bench and Chatbot Arena (2023, neutral): “While our initial evaluations using MT-bench and Chatbot Arena rely on human ratings, collecting human preferences can be costly and laborious [43, 37, 30, 2, 13].”
• LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models (2024, unknown): “LMFlow (Diao et al., 2024) supports training LLMs for specialized domains or tasks.”
• SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuning (2024, unknown): “TheLMFlow[13]encapsulatesmodeltraining processinapipeline style.”
• Iterative Preference Learning from Human Feedback: Bridging Theory and Practice for RLHF under KL-constraint (2023, methodology): “Dong et al. (2023); Yuan et al. (2023); Touvron et al. (2023); Gulcehre et al. (2023) propose reward ranked finetuning (RAFT) (also known as the iterative finetuning, rejection sampling finetuning) by iteratively learning from the best-of-n policy (Nakano et al., 2021) to maximize the reward, which ”
• PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimization (2023, methodology): “Subsequent research [16, 17, 18] have typically adopted Alpaca’s hyperparameters as a standard for training their LLMs.”
• LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models (2023, methodology): “Additionally, the toolbox of fine-tuning LLM, known as LMFlow (Diao et al., 2023), has been developed.”
• RLHF Workflow: From Reward Modeling to Online RLHF (2024, methodology): “The preferences of the responses are annotated by humans.”