Hyperspherical Prototype Networks

By Pascal Mettes, Elise van der Pol, Cees G. M. Snoek. 2019

161 citations recorded by the source corpus.

This paper introduces hyperspherical prototype networks, which unify classification and regression with prototypes on hyperspherical output spaces. For classification, a common approach is to define prototypes as the mean output vector over training examples per class. Here, we propose to use hyperspheres as output spaces, with class prototypes defined a priori with large margin separation. We position prototypes through data-independent optimization, with an extension to incorporate priors from class semantics. By doing so, we do not require any prototype updating, we can handle any training size, and the output dimensionality is no longer constrained to the number of classes. Furthermore, we generalize to regression, by optimizing outputs as an interpolation between two prototypes on the hypersphere. Since both tasks are now defined by the same loss function, they can be jointly trained for multi-task problems. Experimentally, we show the benefit of hyperspherical prototype networks for classification, regression, and their combination over other prototype methods, softmax cross-entropy, and mean squared error approaches.

Subfields: Computer Networks and Communications, Mechanical Engineering, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Mechanical Engineering

In Model Architectures

Citations in the corpus

Making Pre-trained Language Models Better Few-shot Learners (2021, methodology): “Inspired by recent work in prototypical few-shot regression (Mettes et al., 2019), we model the problem as an interpolation between two opposing poles {yl, yu} with values vl and vu, respectively.”
Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere (2020, background): “Mettes et al. (2019) optimizes prototype representations on the unit hypersphere for classification.”
Dissecting Supervised Constrastive Learning (2021, background): “Overall, (Hoffer et al., 2018; Liu et al., 2018; Mettes et al., 2019) all control, in one way or the other, the geometric arrangement of classiﬁer weights and thereby, implicitly, the arrangement of the representations.”
Rethinking Semantic Segmentation: A Prototype View (2022, background): “, cross-entropy loss), ignoring known inductive biases [83, 84], e.”
Regularizing Neural Networks via Minimizing Hyperspherical Energy (2019, background): “The importance of regularizing angular information is also discussed in [31, 32, 33, 34, 35, 36, 37, 38, 39, 40].”
SphereFace Revived: Unifying Hyperspherical Face Recognition (2021, background): “Beyond face recognition, the idea of learning a representation on the hypersphere is also shown generally useful in a diverse set of applications, such as few-shot recognition [47], [48], [49], [50], [51], deep metric learning [25], [52], self-supervised learning [53], [54], [55], [56], generative m”