Learning Strict Identity Mappings in Deep Residual Networks
By Xin Yu, Zhiding Yu, Srikumar Ramalingam. 2018
48 citations recorded by the source corpus.
A family of super deep networks, referred to as residual networks or ResNet [14], achieved record-beating performance in various visual tasks such as image recognition, object detection, and semantic segmentation. The ability to train very deep networks naturally pushed the researchers to use enormous resources to achieve the best performance. Consequently, in many applications super deep residual networks were employed for just a marginal improvement in performance. In this paper, we propose ∊-ResNet that allows us to automatically discard redundant layers, which produces responses that are smaller than a threshold ∊, without any loss in performance. The ∊-ResNet architecture can be achieved using a few additional rectified linear units in the original ResNet. Our method does not use any additional variables nor numerous trials like other hyperparameter optimization techniques. The layer selection is achieved using a single training process and the evaluation is performed on CIFAR-10, CIFAR-100, SVHN, and ImageNet datasets. In some instances, we achieve about 80% reduction in the number of parameters.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Tong Lu
- Zhaoyang Liu
- Yin-Dong Zheng
- Mario A. Nascimento
- Amirreza Lashkari
- Tom Ching LingChen
- Ava Khonsari
- Limin Wang
In Computer Vision and Pattern Recognition
- Tong Lu
- Zhaoyang Liu
- Yin-Dong Zheng
- Limin Wang
- Mario A. Nascimento
- Amirreza Lashkari
- Tom Ching LingChen
- Ava Khonsari
Citations in the corpus
- UniformAugment: A Search-free Probabilistic Data Augmentation Approach (2020, methodology): “We compared UA results with those published by other recent methods, namely AA [3], FAA [12], PBA [8] and RA [4] 3 , as well as Baselines which only include the default augmentations for of the following models: WideResNet [17], Shake-Shake [6], ResNet-50 [7] and ResNet-200 [16].”
- Dynamic Sampling Networks for Efficient Action Recognition in Videos (2020, methodology): “Efficient network architectures proposed recently such as MobileNet [44] and -ResNet [45] were developed for training”