Wanderlust: Online Continual Object Detection in the Real World

By Jianren Wang, Xin Wang, Shang-Guan Yue, Abhinav Kumar Gupta. 2021

75 citations recorded by the source corpus.

Online continual learning from data streams in dynamic environments is a critical direction in the computer vision field. However, realistic benchmarks and fundamental studies in this line are still missing. To bridge the gap, we present a new online continual object detection benchmark with an egocentric video dataset, Objects Around Krishna (OAK). OAK adopts the KrishnaCAM videos, an ego-centric video stream collected over nine months by a graduate student. OAK provides exhaustive bounding box annotations of 80 video snippets (~17.5 hours) for 105 object categories in outdoor scenes. The emergence of new object categories in our benchmark follows a pattern similar to what a single person might see in their day-to-day life. The dataset also captures the natural distribution shifts as the person travels to different places. These egocentric long running videos provide a realistic playground for continual learning algorithms, especially in online embodied settings. We also introduce new evaluation metrics to evaluate the model performance and catastrophic forgetting and provide baseline studies for online continual object detection. We believe this benchmark will pose new exciting challenges for learning from non-stationary data in continual learning. The OAK dataset and the associated benchmark are released at https://oakdata.github.io/.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Meta & Continual Learning

Citations in the corpus

A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “IOD is not only applicable for 2D images, but also for 3D images [503] and videos [425].”
Online Continual Learning through Mutual Information Maximization (2022, methodology): “Wang et al. (2021) proposed to use online CL in object detection.”
Class-Incremental Learning: A Survey (2023, unknown): “…in classification into other tasks, e.g. , semantic segmentation [193], [238], [239], [240], [241], object detection [137], [242], [243], [244], [245], [246], [247], video understanding [91], [161], medical surgery [248], [249], language model [250], [251], [252], [253], language-vision model…”
Deep Class-Incremental Learning: A Survey (2023, unknown): “, semantic segmentation [164], [165], [166], [167], [174], object detection [175], [176], [177], [178], [179], [180], [181], video understanding [95], [133], medical surgery [182], [183], language model [184], [185], [186], [187], language-vision model [188], [189], etc.”
CORA: Benchmarks, Baselines, and Metrics as a Platform for Continual Reinforcement Learning Agents (2021, background): “Isolated Forgetting (F): Isolated Forgetting, originally inspired by (Lopez-Paz & Ranzato, 2017; Wang et al., 2021), represents how much is forgotten from a learned task during later tasks.”