Automatic Differentiation: Techniques and Applications

By L. B. Rall. 1981

Subfields: Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Evaluation & Benchmarks

1. Mengye Ren
2. Roger Baker Grosse
3. Ionel M. Navon
4. Eduardo Alonso
5. Siegfried M. Rump
6. Michael Fairbank
7. Nikolaos V. Sahinidis
8. David Kristjanson Duvenaud

Citations in the corpus

• The Reversible Residual Network: Backpropagation Without Storing Activations (2017, methodology): “We treat backprop as an instance of reverse mode automatic differentiation [24].”
• Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphology (2019, methodology): “We treat the negative log-likelihood − log (Pr(m |T,p)) as the loss function for tree T and compute its gradient using automatic differentiation (Rall, 1981).”
• Backpropagation through the Void: Optimizing control variates for black-box gradient estimation (2017, background): “The back-propagation algorithm (Rumelhart & Hinton, 1986), also known as reverse-mode automatic differentiation (Speelpenning, 1980; Rall, 1981) computes exact gradients of deterministic, differentiable objective functions.”
• Data Assimilation for Numerical Weather Prediction: A Review (2009, methodology): “Earlier books on AD are those by Rall[305] and Kagiwada et al.”
• GEKKO Optimization Suite (2018, background): “Most modern modeling languages leverage automatic differentiation (AD) [9] to facilitate exact gradients without explicit derivative definition by the user.”
• Solving Differential-Algebraic Equations by Taylor Series (I): Computing Taylor Coefficients (2005, background): “Rall [ 29 ] details AD algorithms, generation of TCs, and application to ODEs.”