Network In Network
By Min Lin, Qiang Chen, Shuicheng Yan. 2013
6,739 citations recorded by the source corpus.
We propose a novel deep network structure called "Network In Network" (NIN) to enhance model discriminability for local patches within the receptive field. The conventional convolutional layer uses linear filters followed by a nonlinear activation function to scan the input. Instead, we build micro neural networks with more complex structures to abstract the data within the receptive field. We instantiate the micro neural network with a multilayer perceptron, which is a potent function approximator. The feature maps are obtained by sliding the micro networks over the input in a similar manner as CNN; they are then fed into the next layer. Deep NIN can be implemented by stacking mutiple of the above described structure. With enhanced local modeling via the micro network, we are able to utilize global average pooling over feature maps in the classification layer, which is easier to interpret and less prone to overfitting than traditional fully connected layers. We demonstrated the state-of-the-art classification performances with NIN on CIFAR-10 and CIFAR-100, and reasonable performances on SVHN and MNIST datasets.
Subfields: Computer Networks and Communications, Model Architectures
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Networks and Communications
- Yoshua Bengio
- Kaiming He
- Alan L. Yuille
- Frank Hutter
- Jost Tobias Springenberg
- Philip H. S. Torr
- Ross B. Girshick
- Jason D. Lee
In Model Architectures
- Quoc V. Le
- Yoshua Bengio
- Alan L. Yuille
- Xiaogang Wang
- Kaiming He
- Jiashi Feng
- Dacheng Tao
- Barret Zoph
Citations in the corpus
- Deep Residual Learning for Image Recognition (2015, methodology): “Strong regularization such as maxout [10] or dropout [14] is applied to obtain the best results ([10, 25, 24, 35]) on this dataset.”
- Xception: Deep Learning with Depthwise Separable Convolutions (2016, methodology): “Inception itself was inspired by the earlier NetworkIn-Network architecture [11].”
- Object Detection via a Multi-region and Semantic Segmentation-Aware CNN Model (2015, neutral): “n tv mAP R-CNN [10] with VGG-Net & bbox reg. VOC12 0.792 0.723 0.629 0.437 0.451 0.677 0.667 0.830 0.393 0.662 0.517 0.822 0.732 0.765 0.642 0.337 0.667 0.561 0.683 0.610 0.630 Network In Network [21] VOC12 0.802 0.738 0.619 0.437 0.430 0.703 0.676 0.807 0.419 0.697 0.517 0.782 0.752 0.769 0.651”
- Fast R-CNN (2015, methodology): “2 For the NUS NIN c2000 and BabyLearning meth-ods, there are no associated publications at this time and we could not find information on the ConvNet architectures used; they are unspecified variants of the Network-in-Network design [16].”
- Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification (2015, background): “, increased depth [29, 33], enlarged width [37, 28], and the use of smaller strides [37, 28, 2, 29]), new nonlinear activations [24, 23, 38, 22, 31, 10], and sophisticated layer designs [33, 12].”
- Convolutional neural networks at constrained time cost (2014, methodology): “We have also tested the Network-In-Network (NIN) method [16] on our models.”
- Deeply-Supervised Nets (2014, methodology): “For a fair comparison and clear illustration of the effectiveness of DSN, we match the complexity of our model to the complexity of the network architectures used in [17] and [9] so that there are a comparable number of parameters.”
- Going deeper with convolutions (2014, methodology): “In general, one can view the Inception model as a logical culmination of [12] while taking inspiration and guidance from the theoretical work by Arora et al [2].”
- Very Deep Convolutional Networks for Large-Scale Image Recognition (2014, background): “In spite of a large depth, the number of weights in our nets is not greater than the number ofweights in a more shallow net with larger conv. layer widths and receptive fields (144M weightsin [20]).”
- ImageNet Large Scale Visual Recognition Challenge (2014, methodology): “In the object detection with provided data track, the winning team NUS used the RCNN framework (Gir-shick et al., 2013) with the network-in-network method (Lin et al., 2014a) incorporating improvements of (Howard, 2014).”
- Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition (2014, background): “Hence, the fixedsize constraint comes only from the fully-connected layers, which exist at a deeper stage of the network.”
- Wide Residual Networks (2016, methodology): “We thus experiment with the following combinations (note that the last combination, i.e., B ( 3 , 1 , 1 ) is similar to effective Network-in-Network [18]”
- Local Relation Networks for Image Recognition (2019, background): “2Since 1×1 convolutions do not involve filtering over neighboring pixels, we do not treat them as convolutions in this paper and refer to them as channel transformations [19].”
- Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks (2016, neutral): “Test Error Maxout [6] 11.68% Network in Network [17] 10.41% Deeply Supervised [16] 9.6% ConvPool-CNN-C [26] 9.31% ALL-CNN-C [26] 9.08% our CNN, mean-only B.N. 8.52% our CNN, weight norm.”
- Bag of Tricks for Image Classification with Convolutional Neural Networks (2018, methodology): “Various new architectures have been proposed since then, including VGG [24], NiN [16], Inception [1], ResNet [9], DenseNet [13], and NASNet [34].”
- EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification (2017, background): “inception module [14], [24]–[26] and the residual unit [9], [10].”
- HyperNetworks (2016, background): “Network in Network (Lin et al., 2014) 8.81% FitNet (Romero et al., 2014) 8.39% Deeply Supervised Nets (Lee et al., 2015) 8.22% Highway Networks (Srivastava et al., 2015) 7.72% ELU (Clevert et al., 2015) 6.55% Original Resnet-110 (He et al., 2016a) 6.43% 1.7 M Stochastic Depth Resnet-110 (Huang et…”
- Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning (2016, background): “Some of the next important milestones were Network-innetwork [9] by Lin et al.”
- Unsupervised Representation Learning by Predicting Image Rotations (2018, methodology): “In our CIFAR-10 experiments we implement the RotNet models with Network-In-Network (NIN) architectures (Lin et al., 2013).”