A Better Activation Function for Artificial Neural Networks

By D. Elliott. 1993

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Lifelong Graph Learning (2020, positive): “We take the one-hot vector as the target vector z i , adopt the cross-entropy loss, and use the softsign [11] function σ ( x ) = x / (1+ | x | ) as the non-linear activation.”
Review and Comparison of Commonly Used Activation Functions for Deep Neural Networks (2020, background): “Softsign Activation Function It is worth mentioning a few attempts to upgrade “s”-shaped functions[24], [25].”
Edge Computing with Artificial Intelligence: A Machine Learning Perspective (2022, methodology): “After the input is activated by an activation function, the obtained number is used as the output of the neuron [58].”
Activation Functions: Comparison of trends in Practice and Research for Deep Learning (2018, methodology): “These are achieved using different mathematical functions and some of the early proposals of activation functions, used for neural network computing were explored by Elliott, 1993 as he studied the usage of the AFs in neural network [47].”
Recurrent Neural Networks for Prediction: Learning Algorithms, Architectures and Stability (2001, background): “Another sigmoid was proposed by David Elliott (Elliott 1993) σ(x) = x 1 + |x| (4.”