Regression on Fixed-Rank Positive Semidefinite Matrices: A Riemannian Approach

By Gilles Meyer, Silvere Bonnabel, Rodolphe Sepulchre. 2010

93 citations recorded by the source corpus.

The paper addresses the problem of learning a regression model parameterized by a fixed-rank positive semidefinite matrix. The focus is on the nonlinear nature of the search space and on scalability to high-dimensional problems. The mathematical developments rely on the theory of gradient descent algorithms adapted to the Riemannian geometry that underlies the set of fixed-rank positive semidefinite matrices. In contrast with previous contributions in the literature, no restrictions are imposed on the range space of the learned matrix. The resulting algorithms maintain a linear complexity in the problem size and enjoy important invariance properties. We apply the proposed algorithms to the problem of learning a distance function parameterized by a positive semidefinite matrix. Good performance is observed on classical benchmarks.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

A literature survey of low‐rank tensor approximation techniques (2013, neutral): “Examples include nonlinear conjugate gradient or Newton-like methods on manifolds of low-rank matrices [36, 192, 199, 259, 260] or low-rank tensors [53, 54, 132, 133, 257], see also Section 3.”
A Novel Space-Time Representation on the Positive Semidefinite Cone for Facial Expression Recognition (2017, background): “Conveniently for us, the Riemannian geometry of the space of these matrices, called the positive semidefinite cone S(2, n), was studied in [7, 15, 27, 38].”
A Riemannian geometry with complete geodesics for the set of positive semidefinite matrices of fixed rank (2013, background): “These curves do however have nice properties: they are complete, available in closed-from and it is straightforward to construct a curve connecting to s.p.s.d. matrices, see, e.g., Meyer et al. (2009), Meyer et al. (2010), Bonnabel et al. (2010), Bonnabel & Sepulchre (2010), Sepulchre et al. (2010).”
The geometry of algorithms using hierarchical tensors (2013, background): “A number of large-scale applications [42–46] show that exploiting the smooth structure of the manifold indeed accelerates the algorithms compared to other, more ad-hocways of dealingwith low-rank constraints.”
Building Deep Networks on Grassmann Manifolds (2016, background): “For more details about the geometry of PSD manifolds and its retraction operation, readers are referred to [7, 38, 46, 31, 30, 2].”
Manopt, a matlab toolbox for optimization on manifolds (2013, background): “Meyer et al. (2011) exploit this to propose low-rank algorithms for metric learning.”
Stochastic Gradient Descent on Riemannian Manifolds (2011, methodology): “To illustrate the benefits of the approach on a recent non-lin ear problem, we focus in this section on the most simple algorithm of [22], and we pr ove new convergence results.”