Active Learning for Convolutional Neural Networks: A Core-Set Approach

By Ozan Sener, Silvio Savarese. 2017

2,623 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Efficiency & Compression

Citations in the corpus

ALICE: Active Learning with Contrastive Natural Language Explanations (2020, methodology): “methods to select additional data-points to label from a pool of unlabeled data (Sener and Savarese, 2018; Settles, 2011, 2009).”
Dataset Distillation (2018, background): “Similarly, core-set construction (Tsang et al., 2005; Har-Peled & Kushal, 2007; Bachem et al., 2017; Sener & Savarese, 2018) and instance selection (Olvera-López et al., 2010) methods aim to select a subset of the entire training data, such that models trained on the subset will perform as well as…”
Distilled One-Shot Federated Learning (2020, methodology): “These methods include dataset pruning [1] and core set construction [2, 31, 27], which keep the examples that are measured to be more useful for training and remove the rest.”
Deep Active Learning with a Neural Architecture Search (2018, methodology): “Coreset [25, 9].”
Online Continual Learning on a Contaminated Data Stream with Blurry Task Boundaries (2022, background): “Rainbow memory [5] and OCS [37] are proposed to diversified sampling by regular interval of uncertainty and constructing a ‘Core-Set’ [31], respectively.”
Online Coreset Selection for Rehearsal-based Continual Learning (2021, neutral): “We compare OCS with regularization-based CL methods: EWC [15] and Stable SGD [28], rehearsal-based CL methods using random replay buffer: A-GEM [9] and ER-Reservior [10], coresetbased methods using CL algorithms: Uniform Sampling, k-means Features [31] and k-means Embeddings [36], and coreset-based ”
Coresets via Bilevel Optimization for Continual Learning and Streaming (2020, background): “These include, among others, k-center clustering in last layer embedding [49] and feature space [43], iCaRL’s selection [46] and retaining the hardest-to-classify points [1].”
Machine Unlearning (2019, background): “A relatively simpler problem is that of core-set selection , where the objective is to choose a subset of the dataset that will produce a hypothesis that is as performant as one obtained while using the entire dataset [50], [51].”
Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgetting (2019, background): “Forgetting could also be alleviated with memory-based methods, where certain information regarding learned tasks are stored to help retaining the performance of the learned tasks (see Lopez-Paz et al., 2017; Sener & Savarese, 2018, for example).”
FSD50K: An Open Dataset of Human-Labeled Sound Events (2020, methodology): “this respect, previous work in image recognition evaluates an AL method on two datasets of 10 classes and on CIFAR-100 (of 100 classes) [126].”
A Wholistic View of Continual Learning with Deep Neural Networks: Forgotten Lessons and the Bridge to Active and Open World Learning (2020, methodology): “later used in deep active learning with k-means as the core set algorithm [120].”
A Comprehensive Study of Class Incremental Learning Algorithms for Visual Tasks (2020, methodology): “We also tried exemplar selection techniques inspired from active learning approaches such as: entropy [52], min margin [51], core-set [50], k-means [62].”
Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics (2020, background): “S and Meyer, 2019), such as uncertainty sampling (Lewis and Gale, 1994) which selects (unlabeled) data points, which a model trained on a small labeled subset, has least confidence in, or predicts as farthest (in vector space, based on cosine similarity) (Sener and Savarese, 2018; Wolf, 2011).”
GroupViT: Semantic Segmentation Emerges from Text Supervision (2022, methodology): “For example, few-shot learning [21, 43, 50, 55, 70, 77, 84] and active learning [8, 62, 66, 67, 83] are proposed to perform segmentation with as few pixel-wise labels as possible.”
ViewAL: Active Learning With Viewpoint Entropy for Semantic Segmentation (2019, methodology): “These include random selection (RAND), softmax margin (MAR) [20, 36, 52], softmax confidence [43, 52] (CONF), softmax entropy [18, 52] (ENT), MC dropout entropy [13] (MCDR), Core-set selection [40] (CSET), maximum representativeness [55] (MREP), CEAL entropy [52] (CEAL), and regional MC dropout entr”
Potential, challenges and future directions for deep learning in prognostics and health management applications (2020, background): “Chosen data are labelled by ‘‘querying an oracle’’, which might be a human expert or another method that affords labelling only a subset of the available data (Sener and Savarese, 2017).”
Discriminative Active Learning (2019, methodology): “While their results show signiﬁcant improvements over random sampling for text and speech recognition tasks, our empirical analysis along with the analysis of (Ducoﬀe & Precioso, 2018) show that this method is not suited for image classiﬁcation using CNNs. Finally, (Sener & Savarese, 2018) and (Geif”
Data Selection for Language Models via Importance Resampling (2023, positive): “Following RoBERTa [48],”
Imbalance Problems in Object Detection: A Review (2019, methodology): “The core set approach [151] uses the relative distances of the examples in the feature space to determine redundant samples whereas Vodrahalli et al.”
Learning Loss for Active Learning (2019, methodology): “The uncertainty approach [26, 19, 55, 52, 49, 4] defines and measures the quantity of uncertainty to select uncertain data points, while the diversity approach [45, 37, 15, 5] selects diverse data points that represent the whole distribution of the unlabeled pool.”