Resnet in Resnet: Generalizing Residual Architectures

By Sasha Targ, Diogo Almeida, Kevin Lyman. 2016

1,099 citations recorded by the source corpus.

Residual networks (ResNets) have recently achieved state-of-the-art on challenging computer vision tasks. We introduce Resnet in Resnet (RiR): a deep dual-stream architecture that generalizes ResNets and standard CNNs and is easily implemented with no computational overhead. RiR consistently improves performance over ResNets, outperforms architectures with similar amounts of augmentation on CIFAR-10, and establishes a new state-of-the-art on CIFAR-100.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Dahua Lin
2. Chen Change Loy
3. Jianfei Cai
4. Michael Maire
5. Jiuxiang Gu
6. Amir Shahroudy
7. Bing Shuai
8. Gang Zeng

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Ming-Hsuan Yang
3. Chen Change Loy
4. Dacheng Tao
5. G. Wang
6. Jianfei Cai
7. Xiaogang Wang
8. Bernt Schiele

In Model Architectures

1. Michael Maire
2. Jianfei Cai
3. Jiuxiang Gu
4. Amir Shahroudy
5. Gang Zeng
6. Yanming Miao
7. Chen Change Loy
8. Dahua Lin

Citations in the corpus

• FractalNet: Ultra-Deep Neural Networks without Residuals (2016, methodology): “A standard scheme for data augmentation on CIFAR-10 and CIFAR-100 is widely employed [21, 2, 31, 8, 9, 11, 33].”
• PolyNet: A Pursuit of Structural Diversity in Very Deep Networks (2016, background): “On the other hand, increasing the width of a network has also been shown to be an effective way to improve performance [32, 27, 1].”
• Recent advances in convolutional neural networks (2015, background): “h et al . [126] generalize dropout and networks with stochastic depth, which can be viewed as an ensemble of ResNets, Dropout ResNets, and Stochastic Depth ResNets. The ResNets in ResNets (RiR) paper [127] describes an architecture that merges classical convolutional networks and residual networks, ”
• Residual Networks of Residual Networks: Multilevel Residual Networks (2016, background): “Resnet in Resnet [30] is a generalized residual architecture that combines ResNets and standard CNNs in parallel residual and nonresidual streams.”
• Interleaved Group Convolutions for Deep Neural Networks (2017, neutral): “- 1001 10:2M 4:92 22:71 - ResNet with stochastic depth [13] 110 1:7M 5:25 24:98 1:75 1202 10:2M 4:91 - - Wide ResNet [41] 16 11:0M 4:27 20:43 - 28 36:5M 4:00 19:25 - With dropout 16 2:7M - - 1:64 RiR [36] 18 10:3M 5:01 22:90 - Multi-ResNet [1] 200 10:2M 4:35 20:42 - 26 72M 3:96 19:45 - DenseNet (k =”
• Weighted residuals for very deep networks (2016, background): “Very deep convolutional networks [8,9], especially with residual units, have shown compelling accuracy and nice convergence behaviors on many challenging computer vision tasks [3,10,11].”
• Review of Image Classification Algorithms Based on Convolutional Neural Networks (2021, background): “Resnet in Resnet (RiR) [122] proposed a deep dual-stream architecture that generalizes ResNets and standard CNNs and is easily implemented with no computational overhead.”