Learning Loss for Active Learning

By Donggeun Yoo, In-So Kweon. 2019

826 citations recorded by the source corpus.

The performance of deep neural networks improves with more annotated data. The problem is that the budget for annotation is limited. One solution to this is active learning, where a model asks human to annotate data that it perceived as uncertain. A variety of recent methods have been proposed to apply active learning to deep networks but most of them are either designed specific for their target tasks or computationally inefficient for large networks. In this paper, we propose a novel active learning method that is simple but task-agnostic, and works efficiently with the deep networks. We attach a small parametric module, named ``loss prediction module,'' to a target network, and learn it to predict target losses of unlabeled inputs. Then, this module can suggest data that the target model is likely to produce a wrong prediction. This method is task-agnostic as networks are learned from a single loss regardless of target tasks. We rigorously validate our method through image classification, object detection, and human pose estimation, with the recent network architectures. The results demonstrate that our method consistently outperforms the previous methods over the tasks.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Few-Shot Continual Active Learning by a Robot (2022, background): “Most active learning techniques use uncertainty sampling to request labels for the most uncertain objects [15, 14, 16].”
Replay in Deep Learning: Current Approaches and Missing Biological Elements (2021, neutral): “…closely related to active learning strategies (Cohn, Atlas, & Ladner, 1994; Lin & Parikh, 2017; Wang, Zhang, Li, Zhang, & Lin, 2016; Settles, 2009; Yoo & So Kweon, 2019; Wei, Iyer, & Bilmes, 2015), with the most common selection methods using uncertainty sampling (Lewis & Gale, 1994; Culotta &…”
Online Learning of Objects through Curiosity-Driven Active Learning (2021, background): “In recent years, attempts have been made to develop active learning techniques to reduce the cost of labelling [7], [8], [9], [10], [11].”
Learning Novel Objects Continually Through Curiosity (2021, background): “In recent years, attempts have been made to develop active learning techniques to reduce the cost of labelling [7], [8], [9], [10], [11].”
Semantic Curiosity for Active Visual Learning (2020, positive): “Our problem is more related to that of active learning [45], where an agent actively acquires labels for unlabeled data to improve its model at the fastest rate [24, 44, 57].”
Textual Data Augmentation for Efficient Active Learning on Tiny Datasets (2020, background): “As described by Yoo and Kweon (2019); Sid-diqui et al. (2019), there are three main selection strategies that can be applied to obtain S P : uncertainty-based approaches, diversity-based approaches, and expected model change.”
PseudoClick: Interactive Image Segmentation with Click Imitation (2022, background): “Our idea of predicting FP&FN errors for segmentation is also related to the idea of “prediction loss” proposed in [28], but the two ideas are investigated in significantly different problem settings.”
A Survey of Deep Active Learning (2020, methodology): “as an example, this figure presents a comparison between the traditional uncertainty measurement method [35, 103, 176] and the uncertainty measurement method of synthesizing information in two stages [59, 178, 182] (i.e., the feature extraction stage and task learning stage). of the uncertain sample”
Towards Robust and Reproducible Active Learning using Neural Networks (2020, methodology): “For example, using 20% labeled data of CIFAR10, the difference between RSB performance reported by [32] and [28] is ≈ 13% under identical settings.”
Deep Learning for Geophysics: Current and Future Trends (2021, methodology): “To train a high-precision model using a small amount of labeled data, active learning is proposed to imitate the self-learning ability of human beings (Yoo & Kweon, 2019).”