Orthogonal nonnegative matrix t-factorizations for clustering

By Chris H. Q. Ding, Tao Li, Wei Peng, Haesun Park. Knowledge Discovery and Data Mining, 2006

1,374 citations recorded by the source corpus.

Currently, most research on nonnegative matrix factorization (NMF)focus on 2-factor $X=FG^T$ factorization. We provide a systematicanalysis of 3-factor $X=FSG^T$ NMF. While it unconstrained 3-factor NMF is equivalent to it unconstrained 2-factor NMF, itconstrained 3-factor NMF brings new features to it constrained 2-factor NMF. We study the orthogonality constraint because it leadsto rigorous clustering interpretation. We provide new rules for updating $F,S, G$ and prove the convergenceof these algorithms. Experiments on 5 datasets and a real world casestudy are performed to show the capability of bi-orthogonal 3-factorNMF on simultaneously clustering rows and columns of the input datamatrix. We provide a new approach of evaluating the quality ofclustering on words using class aggregate distribution andmulti-peak distribution. We also provide an overview of various NMF extensions andexamine their relationships.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/edde9337e61139fa3b24aac3dd685d33d6184650

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

The Relationships Among Various Nonnegative Matrix Factorization Methods for Clustering (2006, methodology): “Various extensions and variations of NMF have been proposed recently [13, 14, 15, 23, 27, 3, 30, 33].”
Learning the Shared Subspace for Multi-task Clustering and Transductive Transfer Classification (2009, background): “semisupervised clustering [1] [2] [3] [4] [5] and co-clustering [6] [7] [8] [9].”
Algorithms for nonnegative matrix and tensor factorizations: a unified view based on block coordinate descent framework (2013, methodology): “In the data mining community, variants of NMF such as convex and semi-NMFs [25,75], orthogonal tri-NMF [26], and group-sparse NMF [56] have been proposed, and using NMF for clustering has been shown to be successful [12,57,63].”
Manifold Regularized Discriminative Nonnegative Matrix Factorization With Fast Gradient Descent (2011, neutral): “In order to ensure the learned bases to be parts-based, we penalize (9) by making bases orthogonal [15] (10) where signiﬁes the matrix whose elements are all one.”
Transfer learning in heterogeneous collaborative filtering domains (2013, methodology): “codebook sharing methods of CBT [31] and RMGM [32] can be considered as adaptive and collective extensions of [50,19].”
Predicting positive and negative links in signed social networks by transfer learning (2013, methodology): “We propose to construct generalizable latent features through NMTF by considering both the source and target networks, and then adopt an AdaBoost-like algorithm with instance weighting to train a good classifier.”
Explainable Recommendation via Multi-Task Learning in Opinionated Text Data (2018, methodology): “NMF: Nonnegative Matrix Factorization [8], which is a widely applied latent factor model for recommendation.”
Robust nonnegative matrix factorization using L21-norm (2011, background): “It was shown [5, 8] that NMF essentially solves a matrix clustering problem.”
Robust (Semi) Nonnegative Graph Embedding (2014, background): “One line of approaches is to explicitly impose sparseness constraints, expecting more sparse representation than the original NMF [3], [16]–[18].”
Community discovery using nonnegative matrix factorization (2011, methodology): “Besides the proposed SNMF algorithm, we also implement the Symmetric Tri- Nonnegative Matrix Factorization (STriNMF) (Ding et al. 2006b) algorithm and the Symmetric Convex Coding (SCC) (Long et al. 2007).”
Multi-Task Clustering with Model Relation Learning (2018, methodology): “(18) leads to the following multiplicative update formula When initializing the cluster indicator Y t , we follow the traditional nonnegative matrix factorization methods, i.e., initializing Y t by the k -means method, then setting et al. , 2006].”
Multi-document summarization via sentence-level semantic analysis and symmetric matrix factorization (2008, background): “The 3-factor nonnegative matrix factorization is proposed to simultaneously cluster the rows and the columns of the input data matrix X [8] Note that S provides additional degrees of freedom such that the low-rank matrix representation remains accurate while F gives row clusters and G gives column…”
A Nonlinear Orthogonal Non-Negative Matrix Factorization Approach to Subspace Clustering (2017, methodology): “Related NMF factorizations include convex NMF, orthogonal NMF and kernel NMF [5, 6, 7, 8, 9, 10].”
PAC-Bayesian Analysis of Co-clustering and Beyond (2010, methodology): “This form of symmetric matrix tri-factorization is briefly mentioned in (Ding et al., 2006).”
Transfer Learning with Graph Co-Regularization (2012, methodology): “Using nonnegative matrix tri-factorization (NMTF) [23] as base model, framework (10) is reduced to GTL3 O3 = ∑ π∈Π ∥∥∥Xπ − UπHVTπ ∥∥∥2 F + σ 2 ∑ π∈Π ∥∥∥VTπVπ − I ∥∥∥2 F + λ ∑ π∈Π tr ( U T πL u πUπ ) + γ ∑ π∈Π tr ( V T πL v πVπ ) (16) The problem in Equation (16) can also be solved by an alternating ”
Cross-language web page classification via dual knowledge transfer using nonnegative matrix tri-factorization (2011, methodology): “Because co-clustering the both sides of an input data matrix makes use of the in-terrelatedness between the data points and features, NMF based co-clustering methods usually report superior performance [5,7].”
Generalized Low Rank Models (2014, background): “Some authors [41] have also considered how to obtain a biorthogonal nonnegative matrix factorization, in which both X and Y T have orthogonal columns.”
Nonnegative Matrix Factorization: A Comprehensive Review (2013, methodology): “cally, such applications include text data mining [16], digital”
Transitive Transfer Learning (2015, methodology): “Non-negative Matrix Tri-factorization (NMTF) is a popular and effective technique for data clustering and classiﬁcation [10].”