Learning to Specialize with Knowledge Distillation for Visual Question Answering

39 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Knowledge Distillation: A Survey (2020, background): “iency estimation (Li et al., 2019), image retrieval (Liu et al., 2019d), depth estimation (Pilzer et al., 2019; Ye et al., 2019), visual odometry (Saputra et al., 2019) and visual question answering (Mun et al., 2018; Aditya et al., 2019). Since knowledge distillation in classiﬁcation task is fundam”
Adaptive multi-teacher multi-level knowledge distillation (2020, methodology): “Despite image classiﬁcation, the tasks of visual question answering [14], semantic segmentation [12] and machine translation [24] have been also beneﬁted from distillation learning.”
Towards Oracle Knowledge Distillation with Neural Architecture Search (2019, methodology): “In addition to model compression by learning small and efficient models, the concept of knowledge distillation is often used for other purposes (Chen, Goodfellow, and Shlens 2016; Li and Hoiem 2016; Mun et al. 2018; Noroozi et al. 2018).”
An Empirical Study on Leveraging Scene Graphs for Visual Question Answering (2019, unknown): “es. Indeed, the performance gain between a simple MLP baseline and a complicated attention model is merely within 5 ˘7% [27]. The improvement brought by newly proposed models usually lies within 0:5% [2,6,54]. Some new paradigms thus aim to better understand the intrinsic behavior of models [35]. On”