N2N Learning: Network to Network Compression via Policy Gradient Reinforcement Learning

By Anubhav Ashok, Nicholas Rhinehart, Fares N. Beainy, Kris M. Kitani. 2017

182 citations recorded by the source corpus.

While bigger and deeper neural network architectures continue to advance the state-of-the-art for many computer vision tasks, real-world adoption of these networks is impeded by hardware and speed constraints. Conventional model compression methods attempt to address this problem by modifying the architecture manually or using pre-defined heuristics. Since the space of all reduced architectures is very large, modifying the architecture of a deep neural network in this way is a difficult task. In this paper, we tackle this issue by introducing a principled method for learning reduced network architectures in a data-driven way using reinforcement learning. Our approach takes a larger `teacher' network as input and outputs a compressed `student' network derived from the `teacher' network. In the first stage of our method, a recurrent policy network aggressively removes layers from the large `teacher' model. In the second stage, another recurrent policy network carefully reduces the size of each remaining layer. The resulting network is then evaluated to obtain a reward -- a score based on the accuracy and compression of the network. Our approach uses this reward signal with policy gradients to train the policies to find a locally optimal student network. Our experiments show that we can achieve compression rates of more than 10x for models such as ResNet-34 while maintaining similar performance to the input `teacher' network. We also present a valuable transfer learning result which shows that policies which are pre-trained on smaller `teacher' networks can be used to rapidly speed up training on larger `teacher' networks.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Reinforcement Learning, Efficiency & Compression

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Reinforcement Learning

In Efficiency & Compression

Citations in the corpus

Similarity-Preserving Knowledge Distillation (2019, background): “[1] developed a reinforcement learning approach to learn the student architecture.”
A Comprehensive Survey of Neural Architecture Search (2020, background): “Contrary to widening or deepening the layers of the existing network in EAS [50], N2N learning [51] considers compressing the teacher network by removing or shrinking the layers.”
Once for All: Train One Network and Specialize it for Efficient Deployment (2019, positive): “We also use the knowledge distillation technique after training the largest neural network (Hinton et al., 2015; Ashok et al., 2018; Yu & Huang, 2019b).”
A Survey on Neural Architecture Search (2019, neutral): “Ashok et al. (2018) Ashok et al. (2018) use reinforcement learning to learn two policies which sequentially remove and shrink layers for any given initial architecture with the aim of compressing it.”
Path-Level Network Transformation for Efficient Architecture Search (2018, methodology): “A similar framework has also been incorporated in Ashok et al. (2018) where the transformation operations change from Net2Net operations in Cai et al. (2018) to compression operations.”
AMC: AutoML for Model Compression and Acceleration on Mobile Devices (2018, methodology): “Inspired by them, N2N [2] integrated reinforcement learning into channel selection.”
Knowledge Distillation: A Survey (2020, methodology): “Currently, most KD methods focus on new types of knowledge or distillation loss functions, leaving the design of theteacher–studentarchitecturespoorlyinvestigated(Nowak and Corso 2018; Crowley et al. 2018; Kang et al. 2020; Liu et al. 2019i; Ashok et al. 2018; Liu et al. 2019a).”
Resource-Efficient Neural Architect (2018, background): “Lastly, training a smaller (student) network to mimic a larger (teacher) network, commonly known as distillation [Hinton et al., 2015, Ashok et al., 2017], has gained traction.”
Apprentice: Using Knowledge Distillation Techniques To Improve Low-Precision Network Accuracy (2017, background): “In N2N learning work, Ashok et al. (2017) propose a reinforcement learning based approach for compressing a teacher network into an equally capable student network.”
Fast and Practical Neural Architecture Search (2019, background): “Similar to that of [1], the hard constraints can be relaxed by redeﬁning the reward function as where Acc is the accuracy of the searched network on validation set.”
Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlooks (2020, methodology): “A similar approach is taken by [13] who leans to remove layers of teacher network based on reinforcement learning (RL).”
Residual Knowledge Distillation (2020, background): “…et al. [3] involved adversarial learning into KD by employing a discriminator to tell whether the outputs of S and T are close enough, Ashok et al. [1] exploited reinforcement learning to ﬁnd out the best network structure of S under the guidance of T , and Wang et al. [19], Gao et al. [7]…”
ADC: Automated Deep Compression and Acceleration with Reinforcement Learning (2018, background): “Fine-grained action space can be detrimental to compression agents [3] using discrete action space [58, 44].”
Towards Oracle Knowledge Distillation with Neural Architecture Search (2019, methodology): “Model ensemble (Breiman 1996; Freund, Schapire, and Abe 1999; Guzman-Rivera, Batra, and Kohli 2012; Lee et al. 2017; 2016; 2015; Mun et al. 2018) is a useful technique to boost performance using multiple models trained ∗Both authors contributed equally Copyright c© 2020, Association for the…”
Learnable Embedding Space for Efficient Neural Architecture Compression (2019, methodology): “Our work is also closely related to N2N (Ashok et al., 2018), which searches for a compressed architecture based on a given teacher network using reinforcement learning.”
Resource Constrained Neural Network Architecture Search: Will a Submodularity Assumption Help? (2019, background): “choice is to first train a network and then squeeze or compress it for a target deployment device [2, 9, 12, 37, 40].”
Cross-Resolution Face Recognition via Prior-Aided Face Hallucination and Residual Knowledge Distillation (2019, methodology): “In contrast to the vanilla knowledge distillation methods [1, 2], we further introduced a teacher assistant network to compensate residual error between the transferred knowledge (feature maps) of student and teacher network.”