Improving the learning speed of 2-layer neural networks by choosing initial values of the adaptive weights

By Derrick Nguyen, Bernard Widrow. 1990

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Evolutionary algorithms for hyperparameter tuning on neural networks models (2014, methodology): “This method generate weights that follow a normal distribution (Nguyen and Widrow 1990).”
Numerical assessment of metamodelling strategies in computationally intensive optimization (2012, methodology): “…at the first ANN training step, we conduct 50 independent training trials starting from different solutions initialized through NguyeneWidrow method (Nguyen and Widrow, 1990) and take the best one (the one with lowest error function value e see equation (2) as introduced in Section 3.3.2).”
A review on weight initialization strategies for neural networks (2021, methodology): “Dolezel et al. (2016) have also suggested the weight initialization technique discussed by Nguyen and Widrow (1990) after comparison with the techniques proposed by Sodhi et al. (2014) and Halawa (2014) as the observation showed that this technique did not get stuck in the local minima even after…”
Learning Halfspaces and Neural Networks with Random Initialization (2015, methodology): “To train a classical two-layer neural network, we use the random initialization scheme of Nguyen and Widrow (1990) and the backpropagation algorithm of Møller (1993).”
Neural networks in geophysical applications (2000, methodology): “Even some optimum bounds for r have been established [see, for example, Nguyen and Widrow (1990)].”
Nonlinear Predictability of Stock Returns Using Financial and Economic Variables (1999, background): “This section gives a brief introduction to the initialization algorithm suggested by Nguyen and Widrow (1990). Without loss of generality, consider the NN model”
Review: machine learning techniques applied to cybersecurity (2019, methodology): “The common method is the one proposed by [49], who proposes, basically, choosing the initial values of the weights so the domain of the input space is distributed to the outputs of the network.”
Quantitative measures to evaluate neural network weight initialization strategies (2017, background): “Nguyen and Widrow [6] suggest that since each hidden node computes a slice of the final function that the network tries to approximate, a good approach is to initialize the weights of a node to a uniform random interval dependent on the magnitude of the vector of weights for the node over the input ”
Short-Term Load and Wind Power Forecasting Using Neural Network-Based Prediction Intervals (2014, methodology): “NW method chooses initial weights to distribute the active region of each neuron in the layer approximately evenly across the layer’s input space [36].”
Forecast combinations of computational intelligence and linear models for the NN5 time series forecasting competition (2011, methodology): “Instead of multiple initializations, we used the initialization method of Nguyen and Widrow (1990) (the Matlab function function initnw).”
Smooth function approximation using neural networks (2005, methodology): “When all the known output elements from the training set are grouped in a vector (8) Equation (7) can be written using matrix notation (9) where is an s-dimensional vector composed of the scalar output bias . is a matrix of sigmoid functions evaluated at input-to-node values , each representing the…”
An Aggregation Approach to Short-Term Traffic Flow Prediction (2009, unknown): “Holt developed an ES method, Holt’s two-parameter method [31], which allows for evolving local linear trends in a time series Ht = αyt + (1 − α)(Ht−1 + bt−1) (3) bt = γ(Ht − Ht−1) + (1 − γ)bt−1 (4) ŷt+p = Ht + btp (5) where Ht is the new smoothed value, α is the smoothing constant for the data (0…”