Knowledge Transfer in Deep convolutional Neural Nets
By Steven Gutstein, Olac Fuentes, Eric Andrew Freudenthal. 2007
55 citations recorded by the source corpus.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yoshua Bengio
- Quoc V. Le
- Mohit Bansal
- Ian J. Goodfellow
- Percy Liang
- Sergey Levine
- Kyunghyun Cho
- Alan L. Yuille
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Andrea Vedaldi
- Andrew Zisserman
- Xiaogang Wang
- Trevor Darrell
- Yi Yang
- Jitendra Malik
Citations in the corpus
- Net2Net: Accelerating Learning via Knowledge Transfer (2015, methodology): “A related approach is to add whole new layers while leaving the lower layers fixed (Gutstein et al., 2008).”
- Knowledge Transfer in Deep Block-Modular Neural Networks (2015, neutral): “In work by Gutstein and colleagues [13] a multi-layer convolutional neural net was trained to recognize a set of digits.”
- From generic to specific deep representations for visual recognition (2014, background): “The performance of all tasks saturates at 200K iterations for all the layers and even earlier for some tasks/layers.”
- Toward Training Recurrent Neural Networks for Lifelong Learning (2019, background): “Some work has explored model expansion as a means of transferring knowledge from a small network to a large network to ease the training of deep neural networks (Gutstein, Fuentes, & Freudenthal, 2008; Furlanello, Lipton, Tschannen, Itti, & Anandkumar, 2018).”
- On Training Recurrent Neural Networks for Lifelong Learning (2018, background): “Some works explored model expansion as a means of transferring knowledge from a small network to a large network to ease the training of deep neural networks (Gutstein et al., 2008; Furlanello et al., 2018).”
- Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlooks (2020, background): “Generally speaking, KD is widely regarded as a primary mechanism that enables humans to quickly learn new complex concepts when given only small training sets with the same or different categories [70].”