Non-Discriminative Data or Weak Model? On the Relative Importance of Data and Model Resolution
By Mark Sandler, Jonathan Baccash, Andrey Zhmoginov, Andrew G. Howard. 2019
33 citations recorded by the source corpus.
We explore the question of how the resolution of the input image ("input resolution") affects the performance of a neural network when compared to the resolution of the hidden layers ("internal resolution"). Adjusting these characteristics is frequently used as a hyperparameter providing a trade-off between model performance and accuracy. An intuitive interpretation is that the reduced information content in the low-resolution input causes decay in the accuracy. In this paper, we show that up to a point, the input resolution alone plays little role in the network performance, and it is the internal resolution that is the critical driver of model quality. We then build on these insights to develop novel neural network architectures that we call Isometric Neural Networks. These models maintain a fixed internal resolution throughout their entire depth. We demonstrate that they lead to high accuracy models with low activation footprint and parameter count.
Subfields: Computer Vision and Pattern Recognition, Media Technology, Evaluation & Benchmarks
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Ming-Hsuan Yang
- Jitendra Malik
- Dacheng Tao
- Trevor Darrell
- Xiaogang Wang
- Andrea Vedaldi
- Antonio Torralba
In Media Technology
In Evaluation & Benchmarks
Citations in the corpus
- High-Performance Large-Scale Image Recognition Without Normalization (2021, neutral): “This is arguably consistent with the observations in Sandler et al. (2019), suggesting that model “internal resolution” is a more important quantity to consider in this respect, but we did not explore this direction in further detail.”
- Patches Are All You Need? (2022, background): “× Isotropic MobileNets.”
- Generative Pretraining From Pixels (2020, methodology): “However, our models still slightly underperform Isometric Neural Nets (Sandler et al., 2019), which achieves 70.”