LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models

By Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Yongqiang Ma. Annual Meeting of the Association for Computational Linguistics, 2024

1,832 citations recorded by the source corpus.

Efficient fine-tuning is vital for adapting large language models (LLMs) to downstream tasks. However, it requires non-trivial efforts to implement these methods on different models. We present LlamaFactory, a unified framework that integrates a suite of cutting-edge efficient training methods. It provides a solution for flexibly customizing the fine-tuning of 100+ LLMs without the need for coding through the built-in web UI LlamaBoard. We empirically validate the efficiency and effectiveness of our framework on language modeling and text generation tasks. It has been released at https://github.com/hiyouga/LLaMA-Factory and received over 25,000 stars and 3,000 forks.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/d5d7b26dbdf09737878644530b226a79c9f43bfb

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Shaohui Lin
2. Wenxuan Huang
3. Zijie Zhai
4. Zheyu Ye
5. Shaoshen Cao
6. Fei Zhao
7. Bohan Jia
8. Yao Hu

Citations in the corpus

• MAmmoTH2: Scaling Instructions from the Web (2024, neutral): “The details of the evaluation datasets are in Appendix B.”
• Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models (2025, positive): “5-VL-7B-Instruct [31] as the base model and train it via supervised fine-tuning (SFT) for 2 epochs on 32 NVIDIA H800 80G GPUs for ∼ 10 hours using Llama-Factory framework [51].”
• SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild (2025, positive): “3 We implement with LLaMA-Factory (Zheng et al., 2024) and adopt common hyperparameters of SFT, including 512 examples per batch with a constant learning rate of 1e-5.”
• Fix Before Search: Benchmarking Agentic Query Visual Pre-processing in Multimodal Retrieval-augmented Generation (2026, unknown): “We fine-tune Qwen3-VL-4B-Instruct on V-QPP-Bench using supervised fine-tuning with LoRA (Hu et al., 2022) via the LLaMA Factory framework (Zheng et al., 2024).”
• AfriqueLLM: How Data Mixing and Model Architecture Impact Continued Pre-training for African Languages (2026, unknown): “The release of powerful open-weight models has broadened access to state-of-the-art language technology.”
• This is Going to Sound Crazy, But What If We Used Large Language Models to Boost Automatic Database Tuning Algorithms By Leveraging Prior History? We Will Find Better Configurations More Quickly Than Retraining From Scratch! (2025, unknown): “For drifts, Join Types , Access Meth-ods , and Sort strategies enable Booster to improve over None by allowing it to locally explore during composition to fix conflicts (i.e., degraded queries) and break out of suboptimal solutions.”