IIRC: Incremental Implicitly-Refined Classification

By Mohamed Abdelsalam, Mojtaba Faramarzi, Shagun Sodhani, A. P. Sarath Chandar. 2020

32 citations recorded by the source corpus.

We introduce the "Incremental Implicitly-Refined Classification (IIRC)" setup, an extension to the class incremental learning setup where the incoming batches of classes have two granularity levels. i.e., each sample could have a high-level (coarse) label like "bear" and a low-level (fine) label like "polar bear". Only one label is provided at a time, and the model has to figure out the other label if it has already learned it. This setup is more aligned with real-life scenarios, where a learner usually interacts with the same family of entities multiple times, discovers more granularity about them, while still trying not to forget previous knowledge. Moreover, this setup enables evaluating models for some important lifelong learning challenges that cannot be easily addressed under the existing setups. These challenges can be motivated by the example "if a model was trained on the class bear in one task and on polar bear in another task, will it forget the concept of bear, will it rightfully infer that a polar bear is still a bear? and will it wrongfully associate the label of polar bear to other breeds of bear?". We develop a standardized benchmark that enables evaluating models on the IIRC setup. We evaluate several state-of-the-art lifelong learning algorithms and highlight their strengths and limitations. For example, distillation-based methods perform relatively well but are prone to incorrectly predicting too many labels per image. We hope that the proposed setup, along with the benchmark, would provide a meaningful problem setting to the practitioners.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Han-Jia Ye
2. De-chuan Zhan
3. Da-Wei Zhou
4. Jun Zhu
5. Hang Su
6. Qiwen Wang
7. Ziwei Liu
8. Liyuan Wang

In Computer Vision and Pattern Recognition

1. Han-Jia Ye
2. De-chuan Zhan
3. Jun Zhu
4. Da-Wei Zhou
5. Hang Su
6. Ziwei Liu
7. Bernt Schiele
8. Jun Liu

In Meta & Continual Learning

1. Han-Jia Ye
2. De-chuan Zhan
3. Da-Wei Zhou
4. Liyuan Wang
5. Xingxing Zhang
6. Hang Su
7. Jun Zhu
8. Qiwen Wang

Citations in the corpus

• A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “Besides, other practical challenges have been considered and incorporated, such as multiple labels [215], noisy labels [29], [216], hierarchical granularity [2] and sub-populations [254], mixture of task similarity [207], long-tailed distribution [267], domain alignment [256],”
• Long-Tailed Class Incremental Learning (2022, background): “[1] proposed another realistic CIL setting in which each class can have two granularity levels: each sample could have a high-level (coarse) label and a low-level (fine) label, but only one label is available for each task.”
• Recent Advances of Continual Learning in Computer Vision: An Overview (2021, positive): “[217] proposed a new problem setup called Incremental Implicitly-Refined Classification (IIRC), in which each data instance has a coarse label and a fine label.”
• Understanding Continual Learning Settings with Data Distribution Drift Analysis (2021, background): “The real-concept drifts might also happen if labels get more precise through time (Abdelsalam et al., 2021).”
• Balancing Between Forgetting and Acquisition in Incremental Subpopulation Learning (2022, background): “ISL v.s. General IL (includes CIL [6] and IIRC [1]) : ISL also differs from CIL.”
• Class-Incremental Learning: A Survey (2023, methodology): “Specifically, we use the same network backbone for all CNN-based compared methods, i.e. , ResNet32 [212] for CIFAR100 and ResNet18 for ImageNet.”
• Scaling the Number of Tasks in Continual Learning (2022, background): “These scenarios are usually built with different data distribution shifts [Lesort et al., 2021b] such as new tasks introduce new classes (class-incremental), new domains (domain-incremental), or new labelization of data Abdelsalam et al. [2021].”
• A Metaverse: Taxonomy, Components, Applications, and Open Challenges (2022, background): “[317] developed a standardized benchmark that enables model evaluation in IIRC settings.”
• Continuum: Simple Management of Complex Continual Learning Scenarios (2021, background): “However, recent works challenge some of those common assumptions such as fixed label [6, 2], uncorrelated data stream [5, 42], labeled new data [16], or even no meta-data about tasks and unavailability of information about the future [11].”
• Deep Class-Incremental Learning: A Survey (2023, background): “Learning incremental models with refined concepts [221], [222], [223], [224], [225] is also vital to building real-world learning systems.”