Hierarchical Novelty Detection for Visual Object Recognition

By Kibok Lee, Kimin Lee, Kyle Min, Yuting Zhang, Jinwoo Shin, Honglak Lee. 2018

79 citations recorded by the source corpus.

Deep neural networks have achieved impressive success in large-scale visual object recognition tasks with a predefined set of classes. However, recognizing objects of novel classes unseen during training still remains challenging. The problem of detecting such novel classes has been addressed in the literature, but most prior works have focused on providing simple binary or regressive decisions, e.g., the output would be "known," "novel," or corresponding confidence intervals. In this paper, we study more informative novelty detection schemes based on a hierarchical classification framework. For an object of a novel class, we aim for finding its closest super class in the hierarchical taxonomy of known classes. To this end, we propose two different approaches termed top-down and flatten methods, and their combination as well. The essential ingredients of our methods are confidence-calibrated classifiers, data relabeling, and the leave-one-out strategy for modeling novel classes under the hierarchical taxonomy. Furthermore, our method can generate a hierarchical embedding that leads to improved generalized zero-shot learning performance in combination with other commonly-used semantic embeddings.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Kimin Lee
2. Jinwoo Shin
3. Honglak Lee
4. Kibok Lee
5. Yixuan Li
6. Pedro Morgado
7. Nuno Vasconcelos
8. Sergey Levine

Citations in the corpus

• Overcoming Catastrophic Forgetting With Unlabeled Data in the Wild (2019, methodology): “To achieve this, the model should learn to be confidence-calibrated by learning with a certain amount of OOD data as well as data of the previous tasks [19, 20].”
• A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks (2018, methodology): “For evaluation, similar to [18], we first draw base-new class accuracy curve by adjusting an additional bias to the new class scores, and measure the area under curve (AUC) since averaging base and new class accuracy may cause an imbalanced measure of the performance between base and new classes.”
• Recent Advances in Open Set Recognition: A Survey (2018, background): “Some related work can be found in [170].”
• Solving Long-tailed Recognition with Deep Realistic Taxonomic Classifier (2020, background): “L (Deep-RTC) .181 .50 .899 .619 TD Table 2: Comparisons to hierarchical classiers. Method CIFAR100-LT AWA2-LT ImageNet-LT inference CNN-RNN [28] .379 .882 .514 TD B-CNN [64] .366 .805 .511 TD/BU HND [43] .374 - - TD NofE [2] .373 .770 .463 BU Deep-RTC .397 .894 .529 TD Animal sea star Limonium Plant”
• A Review of Generalized Zero-Shot Learning Methods (2020, methodology): “ries from single label [92], [183], [184], [185], [186], [187] to”
• Generalized Out-of-Distribution Detection: A Survey (2021, unknown): “Some works arrange the large semantic space into a hierarchical taxonomy of known classes (Lee et al., 2018c; Huang et al., 2021; Linderman et al., 2023).”
• Learning to Classify With Incremental New Class (2021, unknown): “As a result, this novel instance is not similar to any one of the known class centers, and the similarity scores to every class prototype tend to be close [43].”
• Incremental Learning with Unlabeled Data in the Wild (2019, methodology): “Also, to make the model confidence-calibrated, a certain amount of OOD data should also be sampled.”