A Tutorial on MM Algorithms

By David R. Hunter, Kenneth L. Lange. 2004

1,873 citations recorded by the source corpus.

Subfields: Management Science and Operations Research, Statistics and Probability, Mathematical Optimization

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Management Science and Operations Research

In Statistics and Probability

In Mathematical Optimization

Citations in the corpus

Interpolated Policy Gradient: Merging On-Policy and Off-Policy Gradient Estimation for Deep Reinforcement Learning (2017, methodology): “Schulman et al. (2015) then uses this property, in combination with minorization-maximization Hunter & Lange (2004), to derive a monotonic convergence proof for a variant of policy iteration algorithm.”
Optimal Transport for Domain Adaptation (2014, methodology): “A classical approach to address this problem is to perform what is called Majorization-minimization [27].”
Learning with Submodular Functions: A Convex Optimization Perspective (2011, background): “ric interpretation of submodular function maximization (left) and optimization of diﬀerences of submodular functions (right). See text for details. dures[210], or majorization-minimization algorithms [104]. They allow the expression of any continuous optimization problem with natural descent algorit”
Adaptive total variation image deblurring: A majorization-minimization approach (2009, methodology): “In recent work [22] [16], we have developed new algorithms to perform image denoising and image deconvolution under TV regularization, which belong to the majorization-minimization (MM) class [24].”
Coordinate descent algorithms for lasso penalized regression (2008, methodology): “The techniques and theory behind MM algorithms deserve to be better known [Hunter and Lange (2004)].”
A New TwIST: Two-Step Iterative Shrinkage/Thresholding Algorithms for Image Restoration (2007, methodology): “IST algorithms for wavelet-based deconvolution were first derived in [41] (see also [28]) under the expectation-maximization (EM) framework and, later [29], using a majorization-minimization (MM, see [33]) approach.”
Learning Triggering Kernels for Multi-dimensional Hawkes Processes (2013, methodology): “To address the problem , we propose MMEL which applies idea of majorization minimization (Hunter and Lange, 2003) to construct an upper-bound of the objective function at each iteration that decouples the parameters so that they can be optimized independently.”
Multi-Stage Multi-Task Learning with Reduced Rank (2016, methodology): “The multi-stage algorithm is an instance of the majorization-minimization (MM) algorithm (Hunter and Lange 2004), an iterative algorithm, which in each iteration constructs a surrogate function as the upper-bound of the original objective function based on the solution of the previous iteration and…”
STRONG ORACLE OPTIMALITY OF FOLDED CONCAVE PENALIZED ESTIMATION (2012, background): “Both LQA and LLA are related to the MM principle (Hunter and Lange, 2004; Hunter and Li, 2005).”
Penalized classification using Fisher’s linear discriminant (2011, methodology): “…deﬁne penalized LDA-FL to be the solution to the problem (13) with a fused lasso penalty (Tibshirani et al., 2005): When the nonnegative tuning parameter λ k is large then the resulting discriminant vector will be sparse in the features, and when the nonnegative tuning parameter γ k is large then…”
A Consolidated Perspective on Multimicrophone Speech Enhancement and Source Separation (2017, neutral): “7) MM Algorithms: EM is a special case of a more general optimization strategy named minorization-maximization (MM) [249].”
A Minorization-Maximization Method for Optimizing Sum Rate in the Downlink of Non-Orthogonal Multiple Access Systems (2015, methodology): “…above, let us define the feasible set, the objective and the set of optimization variables in thetth iteration, respectively, as Ft = [wi for all i, r, w̄,v,A|constraints in(Pbt) are satsified] (31) Ot = max [ N ∑ k=1 rk| {wi for all i, r, w̄,v,A} ∈ Ft ] (32) Vt = [wi for all i, r, w̄,v,A].”
Fast Image Recovery Using Variable Splitting and Constrained Optimization (2009, neutral): “IST can be derived as an expectation-maximization (EM) algorithm [22], as a majorization-minimization (MM, [29]) method [15], [23], or as a forward-backward splitting technique [13], [27].”
Monte-Carlo Sure: A Black-Box Optimization of Regularization Parameters for General Denoising Algorithms (2008, neutral): “majorization-minimization (MM) [45] or half-quadratic [46] optimization].”
A mixture of experts model for rank data with applications in election studies (2008, methodology): “Traditional MoE models, such as those fitted in Jordan and Jacobs (1994), Jacobs et al. (1991) and Peng, Jacobs and Tanner (1996), employ probability densities for the expert networks which are members of the exponential family, that is, the traditional MoE model has the form of a mixture of general”
Hyperspectral Unmixing via $L_{1/2}$ Sparsity-Constrained Nonnegative Matrix Factorization (2011, background): “Multiplicative iterative algorithms, which belong to the majorization-minimization family, have relatively low complexity and can overcome some of these drawbacks [39].”