R&B: Domain Regrouping and Data Mixture Balancing for Efficient Foundation Model Training

By Albert Ge, Tzu-Heng Huang, John Cooper, Avi Trost, Zi-Yi Chu, Satya Sai Srinath Namburi Gnvv, Zi-Yang Cai, Kendall Park, N. Roberts, Frederic Sala. 2025

Subfields: Large Language Models, Optimization & Training