ObjectNet Dataset: Reanalysis and Correction
By Ali Borji. 2020
12 citations recorded by the source corpus.
Recently, Barbu et al introduced a dataset called ObjectNet which includes objects in daily life situations. They showed a dramatic performance drop of the state of the art object recognition models on this dataset. Due to the importance and implications of their results regarding generalization ability of deep models, we take a second look at their findings. We highlight a major problem with their work which is applying object recognizers to the scenes containing multiple objects rather than isolated objects. The latter results in around 20-30% performance gain using our code. Compared with the results reported in the ObjectNet paper, we observe that around 10-15 % of the performance loss can be recovered, without any test time data augmentation. In accordance with Barbu et al.'s conclusions, however, we also conclude that deep models suffer drastically on this dataset. Thus, we believe that ObjectNet remains a challenging dataset for testing the generalization power of models beyond datasets on which they have been trained.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Evaluation & Benchmarks
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Neil Houlsby
- Xiaohua Zhai
- Lucas Beyer
- Alexander Kolesnikov
- Joan Puigcerver
- Michael Tschannen
- Sylvain Gelly
- Mario Lucic
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Neil Houlsby
- Trevor Darrell
- Jitendra Malik
- Ming-Hsuan Yang
- Xiaogang Wang
- Xiaohua Zhai
In Evaluation & Benchmarks
- Neil Houlsby
- Xiaohua Zhai
- Lucas Beyer
- Alexander Kolesnikov
- Sylvain Gelly
- Mario Lucic
- Michael Tschannen
- Jessica Yung
Citations in the corpus
- On Robustness and Transferability of Convolutional Neural Networks (2020, background): “Given this clutter, and arguably better suitability as a detection than recognition task [5], Y |X might be hard to define and the dataset goes beyond a covariate shift.”
- Big Transfer (BiT): General Visual Representation Learning (2019, positive): “ted top-5 accuracy of 80.0%, an almost 25% absolute improvement over the previous state-of-the-art. We provide numeric results and additional results when classifying individual object bounding boxes [6] in the Supplementary Material sectionB. Big Transfer (BiT): General Visual Representation Learni”