Training Student Networks for Acceleration with Conditional Adversarial Networks

By Zheng Xu, Yen-Chang Hsu, Jiawei Huang. 2018

27 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Jianping Gou
2. Dacheng Tao
3. Baosheng Yu
4. S. Maybank
5. Yuan Xie
6. Yanyun Qu
7. Ruishan Liu
8. Nicoló Fusi

Citations in the corpus

• Knowledge Distillation: A Survey (2020, methodology): “y this, many adversarial knowledge distillation methods have been proposed to enable the teacher and student networks to have a better understanding of the true data distribution (Wang et al., 2018d; Xu et al., 2018a; Micaelli and Storkey, 2019; Xu et al., 2018b; Liu et al., 2018b; Wang et al., 2018”
• Adversarial Network Compression (2018, unknown): “, as a tool for transferring knowledge from the teacher to the student by their cooperative exploration. Also dierently from the original idea of Hinton et al. [20] and the recent one from Xu et al. [57], we do not require labels for the studenttraining during compression. In the experiments, our re”
• Knowledge Squeezed Adversarial Network Compression (2019, methodology): “We adjust the teacher network to ResNet-152 like ANC [14] and TSCAN [15] since the pretrain model is available, the student network is changed to ResNet-50/18, the mini-batch size is set to 128, and the total epoch is altered to 120 for ImageNet dataset, where the learning rate is divided by 10 at e”
• Two-stage Image Classification Supervised by a Single Teacher Single Student Model (2019, methodology): “nts problem [12]. The multi-teacher single-student problem was solved by a multi-teacher networks model [13]. Zheng et al. combined GAN with the teacher-student problem and achieved effective results [14]. In our opinion, the teacher-student model is a special case of the two-stage classification. H”
• Teacher-Student Compression with Generative Adversarial Networks (2018, methodology): “These differ from standard TSC by employing non-standard GAN-type compression losses, in which the student acts as the discriminator (Wang et al., 2018b) or generator (Xu et al., 2018; Wang et al., 2018a); Wang et al. (2018a) also train the teacher and student together.”
• Knowledge Distillation for End-to-End Person Search (2019, unknown): “It has gained attention for its promise of more effective training [45], better accuracy [3] and efficiency [2, 41, 49, 58], but it remains strongly limited to networks solving the single classification task.”