Incremental Object Detection via Meta-Learning

By K. J. Joseph, Jathushan Rajasegaran, Salman Hameed Khan, Fahad Shahbaz Khan, Vineeth Nallure Balasubramanian, Ling Shao. 2020

133 citations recorded by the source corpus.

In a real-world setting, object instances from new classes can be continuously encountered by object detectors. When existing object detectors are applied to such scenarios, their performance on old classes deteriorates significantly. A few efforts have been reported to address this limitation, all of which apply variants of knowledge distillation to avoid catastrophic forgetting. We note that although distillation helps to retain previous learning, it obstructs fast adaptability to new tasks, which is a critical requirement for incremental learning. In this pursuit, we propose a meta-learning approach that learns to reshape model gradients, such that information across incremental tasks is optimally shared. This ensures a seamless information transfer via a meta-learned gradient preconditioning that minimizes forgetting and maximizes knowledge transfer. In comparison to existing meta-learning methods, our approach is task-agnostic, allows incremental addition of new-classes and scales to high-capacity models for object detection. We evaluate our approach on a variety of incremental learning settings defined on PASCAL-VOC and MS COCO datasets, where our approach performs favourably well against state-of-the-art methods. Code and trained models: https://github.com/JosephKJ/iOD.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Han-Jia Ye
2. De-chuan Zhan
3. Da-Wei Zhou
4. Jun Zhu
5. Hang Su
6. Ziwei Liu
7. Liyuan Wang
8. Xingxing Zhang

In Computer Vision and Pattern Recognition

1. Han-Jia Ye
2. De-chuan Zhan
3. Jun Zhu
4. Da-Wei Zhou
5. Hang Su
6. Ziwei Liu
7. Liyuan Wang
8. Xingxing Zhang

In Meta & Continual Learning

1. Xingxing Zhang
2. Liyuan Wang
3. Han-Jia Ye
4. Hang Su
5. De-chuan Zhan
6. Jun Zhu
7. Da-Wei Zhou
8. Qiwen Wang

Citations in the corpus

• A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, methodology): “To further mitigate the negative effects of knowledge distillation on learning plasticity, IOD-ML [222] adopts meta-learning to reshape parameter gradients into a balanced direction between the old and new classes.”
• Class-Incremental Learning with Cross-Space Clustering and Controlled Transfer (2022, methodology): “Since the data of the previous tasks are inaccessible, these methods typically enforce distillation objectives on the current data [32, 12], data from an exemplar memory [19, 13, 42, 7, 2, 27, 23], external data [30] or synthetic data [59].”
• Novel Class Discovery without Forgetting (2022, methodology): “Such feature distillation loss has been used in incremental detectors [28] and is simpler than the LessForget constraint from LUCIR [23].”
• Few-Shot Class-Incremental Learning by Sampling Multi-Phase Tasks (2022, background): “W ITH the rapid development of supervised learning, current learning systems have achieved or even surpassed humanlevel performances in many tasks [1], [2], [3], [4], [5], [6].”
• Online Continual Learning in Image Classification: An Empirical Survey (2021, background): “More recently, a meta-learning based approach is proposed to reshape model gradients for better information share across incremental tasks [130].”
• Deep Class-Incremental Learning: A Survey (2023, unknown): “, semantic segmentation [164], [165], [166], [167], [174], object detection [175], [176], [177], [178], [179], [180], [181], video understanding [95], [133], medical surgery [182], [183], language model [184], [185], [186], [187], language-vision model [188], [189], etc.”
• Online Continual Learning For Interactive Instruction Following Agents (2024, background): “Unfortunately, continuously updated instance segmentation models (Joseph et al., 2021; Cermelli et al., 2022) often noticeably underperform jointly trained models.”
• Continual Detection Transformer for Incremental Object Detection (2023, unknown): “Unsurprisingly, forgetting also affects object detection [2, 12, 20, 37, 44, 50, 54].”
• Augmented Box Replay: Overcoming Foreground Shift for Incremental Object Detection (2023, methodology): “Our improvements over the methods storing exemplars [21,27,28] confirm the importance of the box replay for IOD.”
• Class-Incremental Learning: A Survey (2023, methodology): “…to apply the techniques in classification into other tasks, e.g. , semantic segmentation [193], [238], [239], [240], [241], object detection [137], [242], [243], [244], [245], [246], [247], video understanding [91], [161], medical surgery [248], [249], language model [250], [251], [252],…”
• VOS: Learning What You Don't Know by Virtual Outlier Synthesis (2022, background): “Most of them either developed measures to mitigate catastraphic forgetting (Joseph et al., 2020) or used auxiliary information (Rahman et al.”