Algorithms for Nonnegative Matrix Factorization with the β-Divergence

By Cédric Févotte, Jérôme Idier. Neural Computation, 2010

854 citations recorded by the source corpus.

This letter describes algorithms for nonnegative matrix factorization (NMF) with the β-divergence (β-NMF). The β-divergence is a family of cost functions parameterized by a single shape parameter β that takes the Euclidean distance, the Kullback-Leibler divergence, and the Itakura-Saito divergence as special cases (β = 2, 1, 0 respectively). The proposed algorithms are based on a surrogate auxiliary function (a local majorization of the criterion function). We first describe a majorization-minimization algorithm that leads to multiplicative updates, which differ from standard heuristic multiplicative updates by a β-dependent power exponent. The monotonicity of the heuristic algorithm can, however, be proven for β ∈ (0, 1) using the proposed auxiliary function. Then we introduce the concept of the majorization-equalization (ME) algorithm, which produces updates that move along constant level sets of the auxiliary function and lead to larger steps than MM. Simulations on synthetic and real data illustrate the faster convergence of the ME approach. The letter also describes how the proposed algorithms can be adapted to two common variants of NMF: penalized NMF (when a penalty function of the factors is added to the criterion function) and convex NMF (when the dictionary is assumed to belong to a known subspace).

Subfields: Computer Vision and Pattern Recognition, Signal Processing

View Paper: https://www.semanticscholar.org/paper/7ea62a36d627339623270fd7afb493dc9cedae76

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

Single channel speech music separation using nonnegative matrix factorization and spectral masks (2011, background): “In [10], two different cost functions were analyzed.”
Bayesian tuning and bandits : an extensible, open source library for AutoML (2018, methodology): “We use non-negative matrix factorization [13, 8] to decompose dpp-matrix into a n-datasets×num-desired-components matrix (dpp-matrix-decomposed) as shown:”
Learning to Separate Object Sounds by Watching Unlabeled Video (2018, methodology): “Popular approaches include Independent Component Analysis (ICA) [36], sparse decomposition [78], Computational Auditory Scene Analysis (CASA) [21], non-negative matrix factorization (NMF) [23,24,46,69], probabilistic latent variable models [34,61], and deep learning [32,35].”
Nonnegative signal factorization with learnt instrument models for sound source separation in close-microphone recordings (2013, background): “First, the derivative∇θlD of the cost function with respect to θl is expressed as a difference between two positive terms ∇+ θl D and ∇− θl D [46].”
A Unified Algorithmic Framework for Block-Structured Optimization Involving Big Data: With applications in machine learning and signal processing (2015, background): “, [56], where both the concave upper bound and the Jensen’s upper bound (cf.”
Interventional Fairness: Causal Database Repair for Algorithmic Fairness (2019, methodology): “Note that any off-the-shelf NMF algorithm (such as [17]) can be used in Algorithm 2, to minimize the Euclidean distance between Pr and Pr′, the empirical distributions associated to B and B′, respectively.”
Majorization-Minimization Algorithms in Signal Processing, Communications, and Machine Learning (2017, background): “We refer the readers to [27] and [132]–[134] for the details.”
SMF: Drift-Aware Matrix Factorization with Seasonal Patterns (2019, methodology): “Static Matrix Factorization Matrix Factorization (MF) techniques including SVD [17], NMF [15], and pLSI [10] have been widely explored, particularly in collaborative filtering [13, 14, 21].”
MOLI: multi-omics late integration with deep neural networks for drug response prediction (2019, methodology): “(2018) and early integration via non-negative matrix factorization (NMF) (Cichocki and Phan, 2009; Févotte and Idier, 2011), against the single-omics (gene expression) ridge regression method proposed by Geeleher et al.”
BERTopic: Neural topic modeling with a class-based TF-IDF procedure (2022, methodology): “Conventional models, such as Latent Dirichlet Allocation (LDA) (Blei et al., 2003) and Non-Negative Matrix Factorization (NMF) (Févotte and Idier, 2011), describe a document as a bag-of-words and model each document as a mixture of latent topics.”