Fast Kernel Classifiers with Online and Active Learning

By Antoine Bordes, Ş. Ertekin, Jason Aaron Edward Weston, L. Eon Bottou. 2005

724 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Active Bias: Training More Accurate Neural Networks by Emphasizing High Variance Samples (2017, positive): “As reported in the previous studies [42, 6], models can sometimes achieve lower generalization error after being trained with only a subset of actively selected training data.”
Not All Samples Are Created Equal: Deep Learning with Importance Sampling (2018, methodology): “For convex optimization problems, many works (Bordes et al., 2005; Zhao & Zhang, 2015; Needell et al., 2014; Canévet et al., 2016; Richtárik & Takáč, 2013) have taken advantage of the difference in importance among the samples to improve the convergence speed of stochastic optimization methods.”
On multi-class classification through the minimization of the confusion matrix norm (2013, methodology): “It is mostly achieved by either resampling methods for rebalancing the data (e.g. Estabrooks et al. (2004)), and/or by using cost-sensitive methods (e.g. Ting (2000)), or with additional assumptions (e.g. active learning within kernel-based methods Bordes et al. (2005)).”
Adaptivity of averaged stochastic gradient descent to local strong convexity for logistic regression (2013, methodology): “All moments of θ n − θ ∗ 2 and f (¯ θ n ) − f ( θ ∗ ) will be bounded, leading to a sub-exponential behavior.”
Dataset Meta-Learning from Kernel Ridge-Regression (2020, background): “As a specific example, many thousands of support vectors are needed for MNIST classification (Bordes et al. (2005)).”
A unified framework for coordinated multi-arm motion planning (2018, methodology): “The authors of LASVM propose an active learning approach for training set selection which is claimed to be 3 times faster than libSVM and has been tested on datasets of several million points with models of marginally lower Nsv than libSVM(Bordes et al. 2005).”
What Is Machine Learning, Artificial Neural Networks and Deep Learning?—Examples of Practical Applications in Medicine (2023, background): “A machine determines a separating hyperplane that maximises the margin between two classes of training objects [20].”
Active Learning as Non-Convex Optimization (2009, methodology): “The right of Figure 3 gives the standard greedy offline algorithm using the min-margin heuristic (Schohn and Cohn, 2000; Bordes et al., 2005; Tong and Koller, 2001).”
Prediction, learning, and games (2006, methodology): “See, for instance, the pioneering paper of Cohn, Atlas, and Ladner [66], the query by committee algorithm of Freund, Seung, Shamir, and Tishby [116], and the more recent approaches of Campbell, Cristianini, and Smola [45], Tong and Koller [289], and Bordes, Ertekin, Weston, and Bottou [35].”
Capturing Long-Tail Distributions of Object Subcategories (2014, background): “(3) Little traini g data for th rare subcategories.”
Learning from Imbalanced Data (2009, methodology): “Finally, the procedure uses this new training set and all unseen training data to actively retrain the SVM using the LASVM online SVM learning algorithm [101] to facilitate the active learning procedure.”
Extreme Learning Machine for Regression and Multiclass Classification (2012, background): “, Pima Indians diabetes, Statlog Australian credit, Bupa Liver disorders [47], and Banana [49]; 2) data sets with relatively small size and high dimensions, e.”
Support Vector Machines and Kernels for Computational Biology (2008, background): “Approximate solvers that can be trained in linear time without a significant loss of accuracy were also developed [61].”
Training Invariant Support Vector Machines using Selective Sampling (2005, methodology): “Then we describe how to organize feasible direction searches into an online learning algorithm amenable to selective sampling [3].”
Training a Support Vector Machine in the Primal (2007, background): “For this reason, one has to resort to approximations (Bordes et al., 2005; Bakir et al., 2005; Collobert et al., 2002; Tsang et al., 2005).”