Training Neural Networks on Tiny Devices with Integrated Rematerialization and Paging

By Shishir G. Patil, Paras Jain, Ion Stoica, Prabal Dutta, Joseph E. Gonzalez. 2022

Subfields: Efficiency & Compression

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Efficiency & Compression

1. Chuang Gan
2. Song Han
3. Ji Lin
4. Ligeng Zhu
5. Joseph E. Gonzalez
6. Wei-Ming Chen
7. Wei-Chen Wang
8. Ion Stoica

Citations in the corpus

• Efficient Memory Management for Large Language Model Serving with PagedAttention (2023, methodology): “Swapping [23, 42, 55], recomputation [7, 24] and their combination [40] have been utilized to reduce the peak memory of training.”
• On-Device Training Under 256KB Memory (2022, methodology): “Recently, POET [57] utilizes rematerialization and paging to train on microcontrollers, but it relies on a large external memory.”