Rectifier Nonlinearities Improve Neural Network Acoustic Models

By Andrew L. Maas. 2013

7,685 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Model Architectures

Citations in the corpus

Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification (2015, background): “Among these advances, the rectifier neuron [24, 9, 23, 38], e.”
A Style-Based Generator Architecture for Generative Adversarial Networks (2018, methodology): “For our style-based generator ( F in Table 1), we use leaky ReLU [41] with α = 0 .”
Generative Adversarial Networks (2021, neutral): “GANs have various applications in different fields, including computer vision, NLP, and healthcare.”
Generative Adversarial Text to Image Synthesis (2016, methodology): “The description embedding φ(t) is first compressed using a fully-connected layer to a small dimension (in practice we used 128) followed by leakyReLU (Maas et al., 2013) and then concatenated to the noise vector z.”
Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs) (2015, background): “However ReLUs are non-negative and, therefore, have a mean activation larger than zero.”
Reliability and Learnability of Human Bandit Feedback for Sequence-to-Sequence Reinforcement Learning (2018, methodology): “Their outputs are concatenated for each time step, further fed to a 1Dconvolution with max-over-time pooling and subsequently a leaky ReLU (Maas et al., 2013) output layer.”
Learning Particle Physics by Example: Location-Aware Generative Adversarial Networks for Physics Synthesis (2017, methodology): “However, we remain consistent with [7, 35] and use Leaky Rectiﬁed Linear Units [36] throughout both the generator and discriminator.”
Conditional Image Synthesis with Auxiliary Classifier GANs (2016, background): “The discriminator D is a deep convolutional neural network with a Leaky ReLU nonlinearity (Maas et al., 2013).”
Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks (2015, methodology): “Similarly, using a gradient descent on the inputs lets us inspect the ideal image that activates certain subsets of ﬁlters (Mordvintsev et al.).”
VQVAE Unsupervised Unit Discovery and Multi-scale Code2Spec Inverter for Zerospeech Challenge 2019 (2019, methodology): “Batch normalization and LeakyReLU activation are applied to every layer, except the last one before the output prediction.”
Getting to 99% Accuracy in Interactive Segmentation (2020, methodology): “It has six convolutional layers (see Table 4), each followed by the LeakyReLu activation function [38].”
StackGAN: Text to Photo-Realistic Image Synthesis with Stacked Generative Adversarial Networks (2016, background): “The Stage-II GAN completes previously ignored text information to generate more photo-realistic details.”