On the Convergence of Multiplicative Update Algorithms for Nonnegative Matrix Factorization

By Chih-Jen Lin. IEEE Transactions on Neural Networks, 2007

445 citations recorded by the source corpus.

Subfields: Computational Mechanics, Computational Theory and Mathematics, Signal Processing

View Paper: https://www.semanticscholar.org/paper/e91038189dd6421d6f7aef465eb359306f9dd0b3

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computational Theory and Mathematics

In Signal Processing

Citations in the corpus

NeNMF: An Optimal Gradient Method for Nonnegative Matrix Factorization (2012, methodology): “To overcome this problem, Lin [21] proposed a modified MUR that converges to a stationary point.”
Manifold Regularized Discriminative Nonnegative Matrix Factorization With Fast Gradient Descent (2011, background): “Any local minimum must be a stationary point, however, it is difﬁcult to make sure any limit point generated by MUR is a stationary point, Lin [29] showed that MUR for NMF does not guarantee the convergence to a local minimum.”
The Why and How of Nonnegative Matrix Factorization (2014, methodology): “There are several ways to ﬁx this issue, e.g., rewriting the MU as a rescaled gradient descent method –see Equation (6): only entries in the same row interact– and modifying the step length [87], or using a small positive lower bound for the entries of W and H [52, 103]; see also [5].”
Algorithms for Nonnegative Matrix Factorization with the β-Divergence (2010, methodology): “Partial results exist for Euclidean NMF (Lin, 2007a), convergence of multiplicative rules for nonnegative linear regression (i.e., when only one of the two matrices is updated) has been studied in a few cases, see, e.g., (Titterington, 1987; De Pierro, 1993; Eggermont and LaRiccia, 1998), but…”
On Tensors, Sparsity, and Nonnegative Factorizations (2011, unknown): “Lin [34] had a similar observation in the LS case but came up with a different correction technique.”
Exemplar-Based Sparse Representations for Noise Robust Automatic Speech Recognition (2011, result): “While theoretically that does not ensure convergence to a stationary point [47], in our studies with real data we have observed that the algorithm converges sufﬁ-ciently robustly.”
Algorithms and applications for approximate nonnegative matrix factorization (2007, background): “Namely, Lin’s modified algorithm is guaranteed to converge to a stationary point ( Lin, 2005a ).”
Constrained Nonnegative Matrix Factorization for Image Representation (2012, methodology): “" Lin [29] shows that the project gradient method converges faster than the multiplicative update method.”
Marble: high-throughput phenotyping from electronic health records via sparse nonnegative tensor factorization (2014, background): “Several works have dealt with this problem by either avoiding zeros [11], altering the updates [19], or shifting elements into the interior [6].”
Graph Regularized Nonnegative Matrix Factorization for Data Representation (2011, methodology): “Particularly, Lin [30] suggests minor modifications on Lee and Seung’s algorithm which can converge.”
Online Nonnegative Matrix Factorization With Robust Stochastic Approximation (2012, methodology): “Several algorithms including the multiplicative update rule [8], [9], projected gradient descent [10], alternating nonnegative least squares [11], active set methods [12], and NeNMF [13] have been utilized to learn NMF.”
Nonnegative Matrix Factorization: A Comprehensive Review (2013, background): “However, a minor modification in their step size of the gradient descent formula achieves a first-order stationary point [57].”