Multi-class Classification without Multi-class Labels

By Yen-Chang Hsu, Zhaoyang Lv, Joel Schlosser, Phillip Odom, Zsolt Kira. 2019

200 citations recorded by the source corpus.

This work presents a new strategy for multi-class classification that requires no class-specific labels, but instead leverages pairwise similarity between examples, which is a weaker form of annotation. The proposed method, meta classification learning, optimizes a binary classifier for pairwise similarity prediction and through this process learns a multi-class classifier as a submodule. We formulate this approach, present a probabilistic graphical model for it, and derive a surprisingly simple loss function that can be used to learn neural network-based models. We then demonstrate that this same framework generalizes to the supervised, unsupervised cross-task, and semi-supervised settings. Our method is evaluated against state of the art in all three learning paradigms and shows a superior or comparable accuracy, providing evidence that learning multi-class classification without multi-class labels is a viable learning option.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Novel Class Discovery without Forgetting (2022, methodology): “Earlier methods like MCL [25] and KCL [24] for general transfer learning across domains and tasks meta-learn a binary similarity function from the labeled data and use it to discover classes in the unlabeled data.”
Class-incremental Novel Class Discovery (2022, methodology): “Most of the proposed NCD solutions rely on stage-wise [19,20,16] or joint [15,36,11] learning schemes on the labelled and the unlabelled data, with the assumption that structures discovered on the labelled images could be leveraged as a proxy supervision on the unlabelled images.”
A Unified Objective for Novel Class Discovery (2021, methodology): “In addition, they employ rank statistics to identify data pairs that belong to the same class and minimize BCE to bring the network output closer for these pairs similarly to [11].”
Joint Representation Learning and Novel Category Discovery on Single- and Multi-modal Data (2021, methodology): “We further enhance the k-means baseline, KCL [21] and MCL [23] by introducing the same constrative learning used in our framework (see rows 7-10).”
Automatically Discovering and Learning New Visual Categories with Ranking Statistics (2020, methodology): “, 2018), MCL (Hsu et al., 2019) and DTC (Han et al.”
Generalized Category Discovery (2022, methodology): “Earlier methods that could be applied to this problem include KCL [21] and MCL [22], both of which maintain two models trained with labelled data and unlabelled data respectively, for general task transfer learning.”
OpenMix: Reviving Known Knowledge for Discovering Novel Visual Categories in an Open World (2020, methodology): “In Table 2, we compare the proposed method with the state-of-the-art methods, including: K-means [15], KCL [10], MCL [11], DTC [8] and RS [7].”
Centroid Networks for Few-Shot Clustering and Unsupervised Few-Shot Classification (2019, methodology): “Numbers with a star* are those reported in (Hsu et al., 2019).”
Learning to Discover Novel Visual Categories via Deep Transfer Clustering (2019, methodology): “In our experiment on ImageNet, we take the pretrained ImageNet 882 classiﬁcation network of [16] as our initial feature extractor.”
Spacing Loss for Discovering Novel Categories (2022, methodology): “Two-stage Methods Early methods in Novel Class Discovery [13, 15, 16] operate in a phased setting.”
Residual Tuning: Toward Novel Category Discovery Without Labels (2022, unknown): “1) “Isolated Training” methods include KCL [14], MCL [15], DTC [9], and our ResTune.”
Deep Amortized Clustering (2019, methodology): “For DEC, KCL and MCL, we used fully-connected layers with 256 hidden units and 3 layers for both similarity prediction and clustering network.”
Neighborhood Contrastive Learning for Novel Class Discovery (2021, background): “We compare the proposed approach with one classical method (k-means [23]) and four state-of-the-art methods (i.e., KCL [16], MCL [17], DTC [13] and RS [12]).”
Novel Visual Category Discovery with Dual Ranking Statistics and Mutual Knowledge Distillation (2021, methodology): “The KCL [25] and MCL [26] methods were designed for general transfer learning across domains and tasks, which can also be applied for novel category discovery.”