OpenMix: Reviving Known Knowledge for Discovering Novel Visual Categories in an Open World

By Zhun Zhong, Linchao Zhu, Zhiming Luo, Shaozi Li, Yi Yang, Nicu Sebe. 2020

150 citations recorded by the source corpus.

In this paper, we tackle the problem of discovering new classes in unlabeled visual data given labeled data from disjoint classes. Existing methods typically first pre-train a model with labeled data, and then identify new classes in unlabeled data via unsupervised clustering. However, the labeled data that provide essential knowledge are often underexplored in the second step. The challenge is that the labeled and unlabeled examples are from non-overlapping classes, which makes it difficult to build a learning relationship between them. In this work, we introduce Open-Mix to mix the unlabeled examples from an open set and the labeled examples from known classes, where their non-overlapping labels and pseudo-labels are simultaneously mixed into a joint label distribution. OpenMix dynamically compounds examples in two ways. First, we produce mixed training images by incorporating labeled examples with unlabeled examples. With the benefit of unique prior knowledge in novel class discovery, the generated pseudo-labels will be more credible than the original unlabeled predictions. As a result, OpenMix helps preventing the model from overfitting on unlabeled samples that may be assigned with wrong pseudo-labels. Second, the first way encourages the unlabeled examples with high class-probabilities to have considerable accuracy. We introduce these examples as reliable anchors and further integrate them with un-labeled samples. This enables us to generate more combinations in unlabeled examples and exploit finer object relations among the new classes. Experiments on three classification datasets demonstrate the effectiveness of the proposed OpenMix, which is superior to state-of-the-art methods in novel class discovery.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Novel Class Discovery without Forgetting (2022, methodology): “Inspired by the data splits used to evaluate Novel Class Discovery methods [19,61,16], we derive a labeled set and unlabeled set from these datasets.”
Class-incremental Novel Class Discovery (2022, background): “It has been shown that NCD benefits more when the model is trained jointly on the labelled data while using a clustering objective on the unlabelled data [15,37,36,11].”
Novel Visual Category Discovery with Dual Ranking Statistics and Mutual Knowledge Distillation (2021, unknown): “Concurrent to our work, several methods [28, 58, 57, 14] are proposed to improve the performance of novel category discovery from different perspectives, showing promising results.”
A Unified Objective for Novel Class Discovery (2021, methodology): “OpenMix [31] generates virtual samples by mixing the labeled and unlabeled data, which can resist the noisy labels of unlabeled data.”
Semi-Supervised and Unsupervised Deep Visual Learning: A Survey (2022, background): “Theopen-setSSLcallsforintegratingOOD detection [223] or novel class discovery [224] with SSL in a uniﬁed model to advance selective exploitation of noisy unlabeled data.”
Joint Representation Learning and Novel Category Discovery on Single- and Multi-modal Data (2021, background): “performance by augmenting the training samples by mixing the labelled and unlabelled data [52], aggregating pseudopositive pairs with contrastive learning [51], designing a unified objective [12], and leveraging local part information [50].”
Novel Class Discovery in Semantic Segmentation (2021, background): “In addition to better transferring knowledge, another mainstream [8,40,41] is to address NCD by exploring noisy pseudo-labels with contrastive learning and labeled base data.”
Residual Tuning: Toward Novel Category Discovery Without Labels (2022, unknown): “In contrast, the methods in the second group rely on “Joint Training,” through mixing both labeled and unlabeled images to accomplish the objective [16]–[19], [21].”