Batch Normalization has Multiple Benefits: An Empirical Study on Residual Networks

By Soham De, Samuel L. Smith. 2019

6 citations recorded by the source corpus.

Subfields: Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Optimization & Training

1. Alexander Kolesnikov
2. Jessica Yung
3. Joan Puigcerver
4. Sylvain Gelly
5. Xiaohua Zhai
6. Neil Houlsby
7. Lucas Beyer

Citations in the corpus

• Big Transfer (BiT): General Visual Representation Learning (2019, positive): “First, computing BN statistics across large batches has been shown to harm generalization [9].”
• Large Scale Learning of General Visual Representations for Transfer (2019, unknown): “Many known algorithms for training with large batch sizes use Batch Normalization (BN) [19] as a component [10] or even highlight it as the key instrument required for large batch training [7].”