Automatic differentiation

By A. Carle. 2003

865 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

DER: Dynamically Expandable Representation for Class Incremental Learning (2021, methodology): “Implementation Details Our method is implemented with PyTorch [24].”
SS-IL: Separated Softmax for Incremental Learning (2020, methodology): “All the baselines use the Resnet-18 [7] architecture and are implemented using PyTorch framework [10].”
Automatic differentiation in machine learning: a survey (2015, methodology): “5 and Table 4, we demonstrate how gradient methods can benefit from AD, looking at the example of Helmholtz free energy function that has been used in AD literature for benchmarking gradient calculations (Griewank, 1989; Griewank and Walther, 2008).”
A Model-Driven Deep Neural Network for Single Image Rain Removal (2020, background): “The key issue of unrolling the algorithm here is how to represent the two proximal operators involved in (8) and (10) while other operations can be naturally performed with generally used operators in normal networks [34].”
Automatic Guide Generation for Stan via NumPyro (2021, methodology): “For comparison, we also report the results of Stan ADVI with the mean-field and full-rank algorithms.”
Semi-Supervised Semantic Segmentation With Cross-Consistency Training (2020, methodology): “The implementation is based on the PyTorch 1.1 [41] framework.”
Incremental smoothing and mapping (2008, methodology): “I have implemented iSAM in the functional programming language OCaml, using exact, automatic differentiation [62] to obtain the Jacobians.”
4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks (2019, methodology): “4) using C++/CUDA and wrap it with PyTorch [21].”
Accelerating Convolutional Neural Networks via Activation Map Compression (2018, methodology): “Baselines were obtained as follows: LeNet-5 and MobileNet-V1 4 were trained from scratch, while Inception-V3 and ResNet-18/34 were obtained from the PyTorch [49] repository 5 .”
SwiftNet: Real-time Video Object Segmentation (2021, methodology): “All experiments are implemented in PyTorch [18] on 1 NVIDIA P100 GPU, source code will be released upon publication.”
Asymmetric Non-Local Neural Networks for Semantic Segmentation (2019, methodology): “Besides the comparison of the single block efﬁciency, we also provide the whole network efﬁciency comparison with the two most advanced methods, PSANet [48] and DenseA-SPP [36], in terms of inference time ( s ), GPU occupation with batch size set to 1 ( MB ) and the number of parameters ( Million ).”
Vision-Language Pre-Training with Triple Contrastive Learning (2022, methodology): “All of our experiments are performed on 8 NVIDIA A100 GPUs with PyTorch framework [36].”
Optimization Methods for Large-Scale Machine Learning (2016, unknown): “…. . . 25 4.3 SG for General Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 4.4 Work Complexity for Large-Scale Learning . . . . . . . . . . . . . . . . . . . . . . . 33 4.5 Commentary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36”
Gen: a general-purpose probabilistic programming system with programmable inference (2019, methodology): “Probabilistic Inference Gen provides language constructs, DSLs, and inference library routines for state-of-the-art techniques from probabilistic modeling and inference, including generative modeling [32], deep neural networks [1], Monte Carlo inference [2, 16, 38], and numerical optimization [17].”
Learning Memory-Guided Normality for Anomaly Detection (2020, methodology): “All models are trained end-to-end using PyTorch [32], taking about 1, 15 and 36 hours for UCSD Ped2, CUHK Avenue, and Shang-haiTech, respectively, with an Nvidia GTX TITAN Xp.”
Automatic Differentiation to Facilitate Maximum Likelihood Estimation in Nonlinear Random Effects Models (2002, methodology): “In the following, we study a simple example taken from Skaug (2001).”
One-Shot Learning in Discriminative Neural Networks (2017, methodology): “To infer the posterior weights, we experimented with an MCMC-based sampling technique, NUTS [3], and a variational technique, ADVI [7].”