Angular Visual Hardness

By Beidi Chen, Weiyang Liu, Animesh Garg, Zhiding Yu, Anshumali Shrivastava, Anima Anandkumar. 2019

60 citations recorded by the source corpus.

Recent convolutional neural networks (CNNs) have led to impressive performance but often suffer from poor calibration. They tend to be overconfident, with the model confidence not always reflecting the underlying true ambiguity and hardness. In this paper, we propose angular visual hardness (AVH), a score given by the normalized angular distance between the sample feature embedding and the target classifier to measure sample hardness. We validate this score with an in-depth and extensive scientific study, and observe that CNN models with the highest accuracy also have the best AVH scores. This agrees with an earlier finding that state-of-art models improve on the classification of harder examples. We observe that the training dynamics of AVH is vastly different compared to the training loss. Specifically, AVH quickly reaches a plateau for all samples even though the training loss keeps improving. This suggests the need for designing better loss functions that can target harder examples more effectively. We also find that AVH has a statistically significant correlation with human visual hardness. Finally, we demonstrate the benefit of AVH to a variety of applications such as self-training for domain adaptation and domain generalization.

Subfields: Computer Vision and Pattern Recognition, Industrial and Manufacturing Engineering, Ophthalmology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Industrial and Manufacturing Engineering

In Ophthalmology

James M. Rehg

Citations in the corpus

Continual Learning by Modeling Intra-Class Variation (2022, methodology): “To enable an effective modeling of ∆f , we draw inspirations from the observations in (Liu et al., 2017b; 2018b;a; Chen et al., 2020) that geodesic distance on hypersphere is well aligned with perceptual difference, and propose to model the distribution ∆f on the hypersphere in MOCA.”
No Subclass Left Behind: Fine-Grained Robustness in Coarse-Grained Classification Problems (2020, background): “[8] extend these ideas by developing notions of visual hardness, and suggest that better loss function design would be useful for improving the performance of machine learning models on harder examples.”
SphereFace Revived: Unifying Hyperspherical Face Recognition (2021, background): “Feature magnitude is closely related to image quality and semantic ambiguity [8], [72], and such information intuitively seems useful to distinguish different faces.”
Why Do Better Loss Functions Lead to Less Transferable Features? (2020, background): “[12] measure class separation via “angular visual hardness,” the arccosine-transformed cosine similarity between the weight vectors and examples, and suggested an association between this metric and accuracy.”
Parameter Efficient Multimodal Transformers for Video Representation Learning (2020, methodology): “Incorporating contrastive learning (Chen et al., 2020) over the CNN embeddings and training the whole model end-to-end is another promising direction for future work.”
Orthogonal Over-Parameterized Training (2020, background): “[45, 44, 8] propose to learn representations on hypersphere and show that the angular information in neural networks, in contrast to magnitude information, preserves the most semantic meaning.”
CurricularFace: Adaptive Curriculum Learning Loss for Deep Face Recognition (2020, background): “As demonstrated in [5, 10], hard sample mining is also a critical step to further improve the ﬁnal accuracy.”
Learning with Hyperspherical Uniformity (2021, background): “[51, 52, 49, 48, 50, 80, 79, 20, 19, 60, 14, 47] propose to learn representations on hypersphere and show that angular information in neural networks, in contrast to magnitude, preserves the key semantics and is very crucial to generalization.”