The Influence of the Sigmoid Function Parameters on the Speed of Backpropagation Learning

By Jun Han, C. Moraga. 1995

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Semi-analytical approximations to statistical moments of sigmoid and softmax mappings of normal variables (2017, background): “By definition, a sigmoid function is real-valued and differentiable, having a non-negative first derivative which is bell shaped (Han and Moraga, 1995).”
On the Approximation of the Cut and Step Functions by Logistic and Gompertz Functions (2015, methodology): “Logistic functions are often used in artificial neural networks [5], [16], [17], [23].”
Gene2vec: gene subsequence embedding for prediction of mammalian N6-methyladenosine sites from mRNA (2018, unknown): “Fully connected layers with an ReLU activation function and output layer predict the binary classification probability with activation function as follows (Han and Moraga 1995):”
Spiking Neural Networks and Their Applications: A Review (2022, background): “This concept is later extended to utilize continuous activation functions, including the sigmoid [13] or hyperbolic tangent function, to deal with analog inputs and outputs; consequently, this enabled the training of the neural network through a powerful backpropagation algorithm that exploits…”
EMD2FNN: A strategy combining empirical mode decomposition and factorization machine based neural network for stock market trend prediction (2019, background): “There are plenty of studies worked on the selection of activation functions, such as the sigmoid (Han & Moraga, 1995), tanh, Relu (Lecun et al., 2015), PRelu (He et al., 2015b), ELU (Clevert et al., 2015) and110 so on.”
A Classification-Based Surrogate-Assisted Evolutionary Algorithm for Expensive Many-Objective Optimization (2019, methodology): “Finally, the sigmoid function is used as the activation function in the hidden and output layer neurons [64]”
FLightNNs: Lightweight Quantized Deep Neural Networks for Fast and Accurate Inference (2019, background): “, ∂w q i ∂tj , we relax the indicator function д(x , tj ) = 1(x > tj ) to a sigmoid function [12], σ (.”
Active Convolution: Learning the Shape of Convolution for Image Classification (2017, background): “Sigmoid [7] and tanh were the basic activations for the very first neural network.”
Semantic Text Matching for Long-Form Documents (2019, background): “whereFd (·) andFf (·) are two fully-connected hidden layers; ReLU(·) is the activation function for the hidden layer; σ (·) is the logistic sigmoid function for generating probabilistic scores as similarities [17].”
Review of Image Classification Algorithms Based on Convolutional Neural Networks (2021, methodology): “This model uses the sigmoid [98] activation function and uses the softmax classifier in the output layer.”
Data‐driven projection method in fluid simulation (2016, background): “/ [21], which is used to add nonlinear factors.”
THERMOMETER ENCODING: ONE HOT WAY TO RESIST (2017, background): “…et al. (2014), the authors provide evidence that several network architectures, including LSTMs (Hochreiter & Schmidhuber, 1997), sigmoid networks (Han & Moraga, 1995), and maxout networks (Goodfellow et al., 2013), are vulnerable to adversarial examples due to the empirical fact that, when…”
An Exploratory Analysis of Multi-Class Uncertainty Approximation in Bayesian Convolutional Neural Networks (2018, background): “A sigmoid function is a bounded, differentiable real function that is defined for all real inputs and has a non-zero derivative everywhere [Han and Moraga, 1995].”
Activation Functions: Comparison of trends in Practice and Research for Deep Learning (2018, background): “It is a bounded differentiable real function, defined for real input values, with positive derivatives everywhere and some degree of smoothness [16].”