Optimization Transfer Using Surrogate Objective Functions

By Kenneth L. Lange, David R. Hunter, I. Yang. 2000

873 citations recorded by the source corpus.

Subfields: Computational Theory and Mathematics, Management Science and Operations Research, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Theory and Mathematics

In Management Science and Operations Research

In Statistics and Probability

Citations in the corpus

Supervised Dimensionality Reduction via Distance Correlation Maximization (2016, methodology): “The G-MM framework of Parizi et al. [2015] is an extension of the well known MM framework of Lange et al. [2000].”
Incremental Majorization-Minimization Optimization with Application to Large-Scale Machine Learning (2014, methodology): “The principle of successively minimizing upper bounds of the objective function is often called majorization-minimization [35] or successive upperbound minimization [48].”
Robust kernel density estimation (2008, methodology): “KIRWLS can also be viewed as a kind of optimization transfer/majorize-minimize algorithm (Lange et al., 2000; Jacobson & Fessler, 2007) with a quadratic surrogate for ρ.”
Simple and Globally Convergent Methods for Accelerating the Convergence of Any EM Algorithm (2008, positive): “SQUAREM can also accelerate the MM (minorize and maximize) algorithms, which do not invoke the missing-data framework, but are monotone like the EM (Lange et al. , 2000).”
Fast and Robust Archetypal Analysis for Representation Learning (2014, background): “Archetypal analysis indeed produces sparse representations of the data points, by approximating them with convex combinations of archetypes; it also provides a non-negative factorization when the data matrix is non-negative.”
An Overview on the Estimation of Large Covariance and Precision Matrices (2015, neutral): “Such an algorithm is called majorization-minimization algorithm (Lange et al., 2000).”
A Selective Review of Group Selection in High-Dimensional Models. (2012, positive): “Liu and Ye (2010) proposed an algorithm for solving the overlapping group LASSO problem.”
Semantic 3D Reconstruction with Continuous Regularization and Ray Potentials Using a Visibility Consistency Constraint (2016, methodology): “One often successfully utilized strategy is the so called majorizeminimize strategy (for example [18]).”
A Selective Overview of Variable Selection in High Dimensional Feature Space (2009, neutral): “Before we close this section, we remark that with the LLA and LQA, the resulting sequence of target values is always nondecreasing, which is a speciﬁc feature of minorization-maximization (MM) algorithms (Hunter and Lange (2000)).”
Secrets of GrabCut and Kernel K-Means (2015, background): “In general, bound optimizers are iterative algorithms that optimize auxiliary functions (upper bounds) for a given energy E(S) assuming that these auxiliary functions are more tractable than the original difficult optimization problem [18, 32].”
Nonconvex Nonsmooth Low Rank Minimization via Iteratively Reweighted Nuclear Norm (2015, background): “It is worth mentioning that IRNN is actually an instance of Majorize-Minimization (MM) strategy [41].”
Quantile Regression via an MM Algorithm (2000, background): “However, given the success of other MM algorithms in circumventing the need for large matrix inversions (Lange et al., 2000; Erdoğan and Fessler, 1999), we are hopeful that the ideas in this paper will lead to a very fast algorithm for performing quantile regression in high-dimensional parameter spa”
A Tutorial On the design, experimentation and application of metaheuristic algorithms to real-World optimization problems (2021, background): “…on approximation-preserving reduction, i.e., relaxing the goal from ﬁnding the optimal solution to obtaining solutions within some bounded distance from the former [45], surrogate objective functions [46] or dimension reduction procedures (rightly after introduced) might be practical alternatives.”
Iterative Thresholding for Sparse Approximations (2008, neutral): “To solve the optimisation problem min y ‖x − Φy‖22 + λ‖y‖0, (2.1) we derive the following iterative algorithm yn+1 = Hλ0.”
Iterative reconstruction techniques in emission computed tomography (2006, background): “Readers can also find a nice review in Lange et al (2000).”