Knowledge Extraction with No Observable Data

By Jaemin Yoo, Minyong Cho, Taebum Kim, U. Kang. 2019

109 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Jianping Gou
2. Dacheng Tao
3. Baosheng Yu
4. S. Maybank
5. Lin Wang
6. Yuang Liu
7. Zhuangdi Zhu
8. Kuk-Jin Yoon

Citations in the corpus

• Knowledge Distillation: A Survey (2020, methodology): “Specifically, in (Chen et al., 2019a; Ye et al., 2020; Micaelli and Storkey, 2019; Yoo et al., 2019; Hu et al., 2020), the transfer data is generated by a GAN.”
• Progressive Network Grafting for Few-Shot Knowledge Distillation (2020, unknown): “DAFL (Chen et al. 2019) and KEG-NET (Yoo et al. 2019) are proposed to synthesize images by random given labels and then student learns to imitate teacher.”
• Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlooks (2020, methodology): “Yoo [257] 7 7 One Noise + class labels Teacher 7”
• Source-Free Domain Adaptation for Semantic Segmentation (2021, methodology): “Most of the data-free KD methods based on generative adversarial networks [3, 49, 30, 11, 47].”
• Self-Distillation as Instance-Specific Label Smoothing (2020, background): “Recently, data-free distillation, a novel scenario in which the original data for the teacher is unavailable to students, has also been extensively studied [6, 7, 25, 40].”
• Knowledge Distillation in Wide Neural Networks: Risk Bound, Data Efficiency and Imperfect Teacher (2020, background): “[23] and [25] adopt adversarial training to reduce the difference of label distribution between teacher and student.”
• Data-Free Knowledge Distillation for Heterogeneous Federated Learning (2021, background): “Later there emerges data-free KD approaches which aim to reconstruct samples used for training the teacher (Yoo et al., 2019; Micaelli & Storkey, 2019).”