Identity Mappings in Deep Residual Networks

By Kaiming He, X. Zhang, Shaoqing Ren. 2016

11,282 citations recorded by the source corpus.

Deep residual networks have emerged as a family of extremely deep architectures showing compelling accuracy and nice convergence behaviors. In this paper, we analyze the propagation formulations behind the residual building blocks, which suggest that the forward and backward signals can be directly propagated from one block to any other block, when using identity mappings as the skip connections and after-addition activation. A series of ablation experiments support the importance of these identity mappings. This motivates us to propose a new residual unit, which makes training easier and improves generalization. We report improved results using a 1001-layer ResNet on CIFAR-10 (4.62 % error) and CIFAR-100, and a 200-layer ResNet on ImageNet. Code is available at: https://github.com/KaimingHe/resnet-1k-layers.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Ocean Engineering, Model Architectures, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Yoshua Bengio
  2. Quoc V. Le
  3. Mohit Bansal
  4. Sergey Levine
  5. Jianfeng Gao
  6. Percy Liang
  7. Dacheng Tao
  8. Ruslan Salakhutdinov

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Ming-Hsuan Yang
  4. Trevor Darrell
  5. Xiaogang Wang
  6. Jitendra Malik
  7. Yi Yang
  8. Andrew Zisserman

In Ocean Engineering

  1. Yi Yang
  2. Tao A. Xiang
  3. Xiaogang Wang
  4. Wanli Ouyang
  5. Ross B. Girshick
  6. Liang Zheng
  7. Abhinav Kumar Gupta
  8. Kaiming He

In Model Architectures

  1. Quoc V. Le
  2. Yoshua Bengio
  3. Alan L. Yuille
  4. Jiashi Feng
  5. Ruslan Salakhutdinov
  6. Kaiming He
  7. Kyunghyun Cho
  8. Shuicheng Yan

In Optimization & Training

  1. Quoc V. Le
  2. Yoshua Bengio
  3. Frank Hutter
  4. Martin Jaggi
  5. Simon Lacoste-Julien
  6. Sebastian U. Stich
  7. Peter Richtárik
  8. Soham De

Citations in the corpus