Hierarchical Subquery Evaluation for Active Learning on a Graph
By Oisin Mac Aodha, Neill D. F. Campbell, Jan Kautz, Gabriel J. Brostow. 2014
110 citations recorded by the source corpus.
To train good supervised and semi-supervised object classifiers, it is critical that we not waste the time of the human experts who are providing the training labels. Existing active learning strategies can have uneven performance, being efficient on some datasets but wasteful on others, or inconsistent just between runs on the same dataset. We propose perplexity based graph construction and a new hierarchical subquery evaluation algorithm to combat this variability, and to release the potential of Expected Error Reduction. Under some specific circumstances, Expected Error Reduction has been one of the strongest-performing informativeness criteria for active learning. Until now, it has also been prohibitively costly to compute for sizeable datasets. We demonstrate our highly practical algorithm, comparing it to other active learning measures on classification datasets that vary in sparsity, dimensionality, and size. Our algorithm is consistent over multiple runs and achieves high accuracy, while querying the human expert for labels at a frequency that matches their desired time budget.
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Edward Johns
- Oisin Mac Aodha
- Donggeun Yoo
- In-So Kweon
- Gabriel J. Brostow
- Mubarak Shah
- Jianwei Ma
- Siwei Yu
Citations in the corpus
- The Unreasonable Effectiveness of Noisy Data for Fine-Grained Recognition (2015, neutral): “In contrast to our work, the oracle is often simulated with either a simple or no noise model [4, 32, 10, 55].”
- Learning Loss for Active Learning (2019, background): “[5, 16, 32] consider how close a data point is to surrounding data points to choose one that could well propagate the knowledge.”
- Becoming the expert - interactive multi-class machine teaching (2015, methodology): “Our teaching strategy, which we refer to as S eer , takes in-spiration from optimal sampling methods found in Active Learning [36, 45, 29].”