Matrix Differential Calculus

By DRAFT–Version, Suvrit Sra. 2005

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

OptNet: Differentiable Optimization as a Layer in Neural Networks (2017, methodology): “To obtain these derivatives, we differentiate the KKT conditions (sufﬁcient and necessary conditions for optimality) of (2) at a solution to the problem using techniques from matrix differential calculus (Magnus & Neudecker, 1988).”
Variational Inference for Large-Scale Models of Discrete Choice (2007, methodology): “Abbreviating w(μh, h,xht) to wht , an argument using matrix differentials (Magnus and Neudecker 2007) gives the Hessian”
The centred parametrization for the multivariate skew-normal distribution (2008, methodology): “Taking into account the symmetry of Ω , the actual parameter set is θ = (ξ, v(Ω), η), for which we can write −E{d2`(θ)} = 1 2 vec(dΩ)>(Ω−1 ⊗ Ω−1)vec(dΩ)+ 2c1vec(dΩ)>(η ⊗ Ω−1)dξ + (dξ)>[Ω−1 + c2a0ηη>]dξ + c2(dη)> A2dη + 2 (dη)>[c1(Id + Ωηη>)−1 − c2a1η>]dξ = 1 2 (dv(Ω))> D>(Ω−1 ⊗ Ω−1)D(dv(Ω))+ 2c1(dv(”
State-Varying Factor Models of Large Dimensions (2018, unknown): “A proper formulation avoiding the singularity uses vech operators and elimination matrices (See Magnus and Neudecker (1988)) 23Here we use the following result: Lemma 1.”
A Framework for Robust Subspace Learning (2003, methodology): “However there is no unique extension of PCA to multi-linear models; see for example (Carroll and Chang, 1970; Neudecker and Magnus, 1999; Kroonenberg and de Leeuw, 1980).”