Transferring Knowledge to Smaller Network with Class-Distance Loss

By Seungwook Kim, Hyo-Eun Kim. 2017

45 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Jianping Gou
2. Dacheng Tao
3. Baosheng Yu
4. S. Maybank
5. Changxing Ding
6. Chao-qun Hong
7. Jing Zhang
8. Jun Yu

Citations in the corpus

• Knowledge Distillation: A Survey (2020, methodology): “From another perspective, the effectiveness of the soft targets is analogous to label smoothing (Kim and Kim 2017) Fig.”
• Training Student Networks for Acceleration with Conditional Adversarial Networks (2018, methodology): “Variants of knowledge distillation has also been applied to tasks in other domains [5, 23, 30, 33] A recent preprint [17] presented promising preliminary results on CIFAR-10 by learning a small ResNet from a large ResNet.”
• Training Shallow and Thin Networks for Acceleration via Knowledge Distillation with Conditional Adversarial Networks (2017, unknown): “A recent preprint [18] presented promising preliminary results on CIFAR-10 by learning a small ResNet from a large ResNet.”