Bounded activation functions for enhanced training stability of deep neural networks on visual pattern recognition problems

By Shan Sung Liew, M. Hani, R. Bakhteri. 2016

161 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Efficient Defenses Against Adversarial Attacks (2017, methodology): “Notice that CNN + BRELU performs best for a distortion of up to (cid:15) = 0 .”
Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey (2018, background): “Kotler and Wong [96] proposed to learn ReLU-based classifier that show robustness against small adversarial perturbations.”
Classification of Alzheimer’s Disease Based on Eight-Layer Convolutional Neural Network with Leaky Rectified Linear Unit and Max Pooling (2018, methodology): “Therefore, a leaky ReLU (LReLU) [32] was proposed with definition of f lrelu 1⁄4 x x > 0 0:01x otherwise ð6Þ”
Activation functions in deep learning: A comprehensive survey and benchmark (2021, background): “Rectified Linear Unit (ReLU), 2010 [17] No Yes No For negative inputs Leaky ReLU (LReLU), 2013 [34] No Yes No No Parametric ReLU (PReLU), 2015 [35] Yes Yes No No Randomized ReLU (RReLU), 2015 [35] No Yes No No Concatenated ReLU (CReLU), 2016 [36] No Yes No For negative inputs Bounded ReLU (BReLU), 2”