A feasible method for optimization with orthogonality constraints

By Zaiwen Wen, Wotao Yin. 2012

971 citations recorded by the source corpus.

Subfields: Computational Mechanics, Numerical Analysis, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Numerical Analysis

In Signal Processing

Citations in the corpus

A PAC-Bayesian bound for Lifelong Learning (2013, methodology): “For the ELLA algorithm, we use the same procedure to set the regularization strength µ , the remaining parameters we leave at their default values.”
Bayesian group factor analysis with structured sparsity (2016, methodology): “This was done by the constraint preserving updates of the objective function (Wen and Yin, 2013 ).”
Notes on Optimization on Stiefel Manifolds (2011, background): “Now that we have some understanding of the tangent spaces of Stiefel manifolds, we can return to the optimization problem of equation(1).”
Efficient Riemannian Optimization on the Stiefel Manifold via the Cayley Transform (2020, methodology): “Also, Wen & Yin (2013) reduce the cost of the Cayley transform by making the restrictive assumption that the matrix size n× p is such that 2p n.”
Orthogonal Weight Normalization: Solution to Optimization over Multiple Dependent Stiefel Manifolds in Deep Neural Networks (2017, methodology): “To solve OMDSM problem, one straightforward idea is to use Riemannian optimization method that is extensively used for single manifold or multiple independent manifolds problem, either in optimization communities (Absil and Malick 2012; Wen and Yin 2013; Cunningham and Ghahramani 2015) or in applica”
Unifying Virtual and Physical Worlds (2017, methodology): “It is shown that the iterative solver of our proposed CLEVER, namely the curvilinear search with Barzilar-Borwein step method, can guarantee the iterations to converge to a stationary point [Wen and Yin 2013].”
Generalized Domain-Adaptive Dictionaries (2013, methodology): “We solved this problem using the efficient approach presented in [25].”
Learning Compact Binary Face Descriptor for Face Recognition (2015, methodology): “In this section, we first present the CBFD feature learning method and then introduce how to use CBFD for face representation.”
Robust Unsupervised Cross-modal Hashing for Multimedia Retrieval (2020, methodology): “…as Setting gradient of Equation (20) with respect to W to zero, we can get where According to the orthogonal constraint optimization procedure [35], we can define a skew-symmetric matrix M = GW T − G T W and update W via Crank-Nicolson-like scheme [28] as follows: where η is the step size, τ…”
Unsupervised Anomaly Detection With LSTM Neural Networks (2017, methodology): “For the LSTM parameters, we use the method for optimization with orthogonality constraints in [31] due to (31).”
Jointly Learning Heterogeneous Features for RGB-D Activity Recognition (2015, methodology): “We provide our proof in the supplementary file, available online, based on some tricks provided in [42].”
Cascaded Projection: End-To-End Network Compression and Acceleration (2019, methodology): “There are several optimization methods on Grassmann manifolds, most of which include iterative optimization and retraction methods [7, 1, 48, 2, 51].”
Spectral Clustering with Graph Neural Networks for Graph Pooling (2019, methodology): “These algorithms search the solution on the manifold induced by the orthogonality constraint on the columns of Q , by performing gradient updates along the geodesics (Wen & Yin, 2013; Collins et al., 2014).”
Example-based 3D object reconstruction from line drawings (2012, methodology): “Then in step 2, we fix t and α and update R using the method in [25], which uses a gradient descent algorithm to find the minimizer of a differentiable objective function subject to an orthogonal constraint.”
Multimodal Manifold Analysis by Simultaneous Diagonalization of Laplacians (2015, neutral): “Alternatively, recent techniques [36], [37] for optimization on Stiefel manifold VkðRk Þ 1⁄4 fA 2 Rk k : A>A 1⁄4 Ig can be employed.”