Distributed Distillation for On-Device Learning

By Ilai Bistritz, Ariana J. Mann, N. Bambos. 2020

96 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Jianping Gou
2. Murali Annavaram
3. Dacheng Tao
4. Baosheng Yu
5. Chaoyang He
6. S. Avestimehr
7. S. Maybank
8. Changxing Ding

Citations in the corpus

• Knowledge Distillation: A Survey (2020, background): “…are other special interesting applications of knowledge distillation, such as neural architecture search (Macko et al. 2019; Bashivanetal.2019),interpretabilityofdeepneuralnetworks (Liu et al. 2018b), and federated learning (Bistritz et al. 2020; Lin et al. 2020; Seo et al. 2020; He et al. 2020a).”
• Group Knowledge Transfer: Federated Learning of Large CNNs at the Edge (2020, background): “pts to break the computational constraint, but it requires frequent communication with the server. Knowledge Distillation (KD). We use KD [19] in a different manner from existing and concurrent works [38, 39, 40, 41, 42, 43, 44, 45]. Previous works only consider transferring knowledge from a large n”