Stream-51: Streaming Classification and Novelty Detection from Videos

By Ryne Roady, Tyler L. Hayes, Hitesh U Vaidya, Christopher Kanan. 2020

28 citations recorded by the source corpus.

Deep neural networks are popular for visual perception tasks such as image classification and object detection. Once trained and deployed in a real-time environment, these models struggle to identify novel inputs not initially represented in the training distribution. Further, they cannot be easily updated on new information or they will catastrophically forget previously learned knowledge. While there has been much interest in developing models capable of overcoming forgetting, most research has focused on incrementally learning from common image classification datasets broken up into large batches. Online streaming learning is a more realistic paradigm where a model must learn one sample at a time from temporally correlated data streams. Although there are a few datasets designed specifically for this protocol, most have limitations such as few classes or poor image quality. In this work, we introduce Stream-51, a new dataset for streaming classification consisting of tempo- rally correlated images from 51 distinct object categories and additional evaluation classes outside of the training distribution to test novelty recognition. We establish unique evaluation protocols, experimental metrics, and baselines for our dataset in the streaming paradigm1.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Anomaly & Outlier Detection

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jun Zhu
2. Hang Su
3. Irina Rish
4. Timothée Lesort
5. Massimo Caccia
6. Xingxing Zhang
7. Liyuan Wang
8. Sarath Chandar

In Computer Vision and Pattern Recognition

1. Jun Zhu
2. Hang Su
3. Liyuan Wang
4. Xingxing Zhang
5. Timothée Lesort
6. Massimo Caccia
7. Irina Rish
8. Sarath Chandar

In Anomaly & Outlier Detection

1. Aaron C. Courville

Citations in the corpus

• A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “There are also many benchmarks for video classification [275], [324], [363], [418], varying in size and purpose.”
• Continuous Coordination As a Realistic Scenario for Lifelong Learning (2021, background): “There have been efforts to address this by proposing more challenging benchmarks like CORe50 (Lomonaco & Mal-toni, 2017), CRIB (Stojanov et al., 2019), OpenLoris (Shi et al., 2020), Stream51 (Roady et al., 2020), and IIRC (Ab-delsalam et al., 2020).”
• A Multi-Head Model for Continual Learning via Out-of-Distribution Replay (2022, unknown): “(Roady et al., 2020) also proposed a new dataset.”
• Understanding Continual Learning Settings with Data Distribution Drift Analysis (2021, methodology): “In continual learning various of them can be used such as Split Mnist/Cifar/ImageNet, Permutation Mnist and more recent ones such as Core50 [45] or Stream-51 [56].”
• Very Fast Streaming Submodular Function Maximization (2020, methodology): “We use two real-world data-sets and one artificial data-set depicted in the bottom group in Table 2: The stream51 dataset [25] contains image frames from a sequence of videos, where each video shows an object of one of 51 classes.”