Multiple-Instance Active Learning

By Burr Settles, Mark W. Craven, Soumya Ray. 2007

669 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

A Survey on Data Collection for Machine Learning: A Big Data - AI Integration Perspective (2018, background): “Here the objective could be to choose an example that maximizes the estimated model accuracy [108].”
The Unreasonable Effectiveness of Noisy Data for Fine-Grained Recognition (2015, methodology): “This strategy is similar to the “expected model change” sampling criteria [43], but done for each class independently.”
Automated Curriculum Learning for Neural Networks (2017, methodology): “Taking this step to be the negative gradient −∇ θ L ( x , θ ) we obtain the gradient prediction gain This measures the magnitude of the gradient vector, which has been used an indicator of data salience in the active learning literature (Settles et al., 2008).”
Identifying Unknown Unknowns in the Open World: Representations and Policies for Guided Exploration (2016, background): “…(e.g., uncertainty sampling (Lewis and Gale 1994; Settles 2009), query by committee (Seung, Opper, and Som-polinsky 1992), expected model change (Settles, Craven, and Ray 2008), expected error reduction (Zhu, Lafferty, and Ghahramani 2003), expected variance reduction (Zhang and Oles 2000)).”
Multi-instance learning by treating instances as non-I.I.D. samples (2008, background): “Recently, research on multi-instance semi-supervised learning [19], multi-instance active learning [24] and multi-instance multi-label learning [37] have also been reported.”
Multi-instance multi-label learning (2008, methodology): “Multi-instance learning techniques have already been applied to diverse applications including image categorization [17, 18], image retrieval [71, 84], text categorization [3, 60], web mining [86], spam detection [37], computer security [54], face detection [66, 76], computer-aided medical diagnosis”
From Theories to Queries (2011, methodology): “Furthermore, recent empirical studies—some employing live active learning with real annotators “in the loop”—have found puzzlingly mixed or negative results (Schein and Ungar, 2007; Guo and Schuurmans, 2008; Settles et al., 2008a; Baldridge and Palmer, 2009).”
Learning Loss for Active Learning (2019, methodology): “Expected model change [44, 48, 12] selects data points that would cause the greatest change to the current model parameters or outputs if we knew their labels.”
Results of the Active Learning Challenge (2011, background): “Since discriminative probabilistic models are usually trained with gradient-based optimization, the “change” imparted can be measured by the magnitude of the gradient (Settles et al., 2008a).”
Active Learning for Coreference Resolution (2012, background): “Several uncertainty metrics have been proposed that follow that route to adapt single instance selection to multiple instance scenarios (Settles et al., 2008; Tomanek et al., 2009).”
Non-I.I.D. Multi-Instance Dimensionality Reduction by Learning a Maximum Bag Margin Subspace (2010, background): “The framework of multi-instance has attracted much attention in various domains such as object detection (Chen and Wang 2004), information retrieval (Settles, Graven, and Ray 2008), image classification (Qi et al. 2007), and biomedical informatics (Fung et al. 2007).”
A survey on instance selection for active learning (2012, methodology): “– Expected Gradient Length focuses on querying instances that cause the maximal change to the current model [46].”
Robust Active Label Correction (2018, methodology): “This criterion was suggested by Settles et al. (2008) in the context of multiple-instance active learning.”
Perceptual Annotation: Measuring Human Vision to Improve Computer Vision (2014, background): “Active learning [1] is a prominent example of such an approach, wherein training set quality is enhanced by placing human annotators “in-the-loop” with a machine learning system [7], [8], [9], [10], [11], [12], [13].”
Active Learning Literature Survey (2009, methodology): “Settles et al. (2008a) propose a novel approach to cost-sensitive active learning in settings where annotation costs are variable and not known, e.g., when the labeling cost is a function of elapsed annotation time.”
A Survey of Deep Active Learning (2020, background): “n a given set of unlabeled datasets, the main query strategies include the uncertainty-based approach [14, 76, 95, 123, 142, 161], diversity-based approach [18, 47, 55, 111] and expected model change [43, 127, 141]. In addition, many works have also studied hybrid query strategies [10, 146, 177, 183”
Discriminative Active Learning (2019, methodology): “(Settles et al., 2008) and (Huang et al., 2016) use this approach by calculating the expected gradient length (EGL) of the loss with respect to the model parameters, when the expectation is over the posterior distribution over the labels induced by the trained classiﬁer.”