R&B: Breaking the Data Mixing Bottleneck with Just 0.01% Overhead
By Albert Ge, Tzu-Heng Huang, John Cooper, Avi Trost, Zi-Yi Chu, S. Srinath, Namburi Gnvv, Zi-Yang Cai, Kendall Park, Nicholas Roberts and 1 more
Subfields: Large Language Models, Optimization & Training