Optimization algorithms on matrix manifolds

By L. Tunçel. 2009

3,246 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Optimization & Training

Citations in the corpus

Revisiting Natural Gradient for Deep Networks (2013, methodology): “Absil et al. (2008) describes how second order methods can be generalized to the manifold case. In Honkela et al. (2008, 2010) a similar idea is proposed in the context of variational inference and specific assumptions on the form of p(θ) are made. Gonzalez and Dorronsoro (2006) is more similar in s”
A literature survey of low‐rank tensor approximation techniques (2013, neutral): “There is a growing interest in applying so called Riemannian optimization techniques [1] to (8).”
Parseval Networks: Improving Robustness to Adversarial Examples (2017, methodology): “There exist several pullback operators for embedded submanifolds such as the Stiefel manifold based for example on Cayley transforms (Absil et al., 2009).”
Continual Learning in Low-rank Orthogonal Subspaces (2020, positive): “The optimization of a differentiable cost function over a Stiefel manifold has been extensively studied in literature [Absil et al., 2009, Bonnabel, 2013].”
Deep Learning on Lie Groups for Skeleton-Based Action Recognition (2016, background): “As studied in [1], the steepest descent direction for the used loss function L(Rk−1, y) with respect to Wk on the manifold SO3 is the Riemannian gradient ∇̃L Wk , which can be obtained by parallel transporting the Euclidean gradients onto the corresponding tangent space.”
Sorting out Lipschitz function approximation (2018, methodology): “Given an input matrix A 0 = A , the algorithm computes, where Importantly, this algorithm is fully differentiable and thus has a pullback operator for the Stiefel manifold [1] allowing us to optimize over orthonormal matrices directly.”
Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age (2016, neutral): “rotations), which are of interest in robotics [1], [83].”
A Survey on Metric Learning for Feature Vectors and Structured Data (2013, methodology): “The optimization is carried out efficiently using recent advances in optimization over Riemannian manifolds (Absil et al., 2008) and based on the low-rank factorization of M .”
McGan: Mean and Covariance Feature Matching GAN (2017, methodology): “We maintain clipping on ω to ensure boundedness of Φω , and perform a QR retraction on the Stiefel manifoldOm,k (Absil et al., 2007), maintaining orthonormality of U and V .”
Geometric Robustness of Deep Networks: Analysis and Improvement (2017, neutral): “The convergence analysis of such methods can be found for example in [1].”
MISO Wireless Communication Systems via Intelligent Reflecting Surfaces : (Invited Paper) (2019, methodology): “More background on manifolds can be found in [15].”
A Riemannian Optimization Approach for Computing Low-Rank Solutions of Lyapunov Equations (2010, methodology): “It is used both in the general context of retraction-based Riemannian optimization (see [3] and the references therein) and in the specific context of low-rank solvers for matrix equations (see [20] and [19]).”
Tensor product methods and entanglement optimization for ab initio quantum chemistry (2014, neutral): “7 T((10)) i x T((7)) j : ↓ i ,↑ j ==> ↑ i ,↓ j : ρ i,j (9,8)”
Toward Dual-functional Radar-Communication Systems: Optimal Waveform Design (2017, methodology): “Noting that the SDR approach solves the problem at the price of high computational costs, we propose in this subsection an Riemannian Conjugate Gradient (RCG) algorithm [51], [52], which finds a near-optimal solution within much lower complexity.”
Projection-like Retractions on Matrix Manifolds (2012, methodology): “3], the exponential update is replaced by any (first-order) retraction, then local quadratic convergence is preserved [4, 2].”
Discriminative clustering for image co-segmentation (2010, methodology): “In order to ﬁnd a local minimum on this quotient space, we can thus use a second-order trust-region method for such manifolds 1 , with guaranteed convergence to local minima rather than stationary points (see, e.g., [1]).”
Understanding symmetries in deep networks (2015, methodology): “…descent direction for a loss (cid:96) ( w ) with w on the unit-norm manifold is computed ˜ ∇ (cid:96) ( w ) = ∇ (cid:96) ( w ) − ( w T ∇ (cid:96) ( w )) w , where ∇ (cid:96) ( w ) is the Euclidean gradient and ˜ ∇ (cid:96) ( w ) is the Riemannian gradient on the unit-norm manifold [8, Chapter 3].”
Legged Robot State-Estimation Through Combined Forward Kinematic and Preintegrated Contact Factors (2017, methodology): “We first establish the mathematical notation where we assume readers are already familiar with basics of Lie groups, Lie Algebra, and optimization on matrix Lie groups [19], [20], [21].”