A comparison of numerical optimizers for logistic regression

By Tom Minka. 2004

297 citations recorded by the source corpus.

View Paper: https://www.semanticscholar.org/paper/d9a53a7108ca6c715d572fcebb567895b190a4cb

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Discriminative Batch Mode Active Learning (2007, methodology): “Logistic regression is a robust classifier that can be trained efficiently using various convex optimization techniques (Minka, 2003).”
Recent Advances of Large-Scale Linear Classification (2012, background): “An early comparison on small-scale data is [29].”
Bundle Methods for Regularized Risk Minimization (2010, methodology): “These methods are effective and efficient even when m and d are large (Sha and Pereira, 2003; Minka, 2007).”
Differentially private distributed logistic regression using private and public data (2014, methodology): “Due to its efficiency in handling convex functions, (i.e., usually only a few iterations (five or six) are needed to reach a very high precision [24]), the Newton method is a popular numerical approach for building a logistic regression model [25].”
Recursive regularization for large-scale classification with hierarchical and graphical dependencies (2013, background): “If the data is highly correlated or dense in the dimensions, they take a much longer time to converge [25] .”
Large-scale sparse logistic regression (2009, methodology): “Standard optimization algorithms such as Newton method and conjugate gradient method [3, 23] can be applied for solving such a formulation.”
A discrete particle swarm optimization method for feature selection in binary classification problems (2010, unknown): “(2008), Minka (2003), and Winker and Gilli (2004).”
Learning Trans-Dimensional Random Fields with Applications to Language Modeling (2018, methodology): “In practice the gradient methods are shown to be much faster than the lower bound methods [26], [27].”
Distributed Newton Methods for Regularized Logistic Regression (2015, methodology): “The unconstrained problem (1) has a twice-differentiable objective function and can be solved by various optimization algorithms; see, for example, [Minka, 2003; Komarek and Moore, 2005; Lin, Weng, and Keerthi, 2008; Huang et al., 2010].”
Density-ratio matching under the Bregman divergence: a unified framework of density-ratio estimation (2012, methodology): “(11) is convex, the global optimal solution can be obtained by a standard non-linear optimization technique such as the gradient descent method or (quasi-)Newton methods (Hastie et al., 2001; Minka, 2007).”
Scalable training of L1-regularized log-linear models (2007, methodology): “…objective function cannot be minimized with generalpurpose gradient-based optimization algorithms such as the l-bfgs quasi-Newton method (Nocedal & Wright, 1999), which has been shown to be superior at training large-scale L2-regularized log-linear models by Malouf (2002) and Minka (2003).”
Anomaly Detection and Localization in Crowded Scenes (2014, methodology): “In each trial, the logistic classifier was trained by Newton’s method [42].”
Deep learning for visual understanding (2017, unknown): “The optimal parameter w is obtained by optimizing the conditional log-likelihood function[245]:”