What Size Net Gives Valid Generalization?

By Eric B. Baum, David Haussler. 1989

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Deep learning in neural networks: An overview (2014, background): “…the MDL principle suggests that low NN weight complexity corresponds to high NN probability in the Bayesian view (e.g., Buntine & Weigend, 1991; De Freitas, 2003; MacKay, 1992; Neal, 1995), and to high generalization performance (e.g., Baum & Haussler, 1989), without overfitting the training data.”
Toward Deeper Understanding of Neural Networks: The Power of Initialization and a Dual View on Expressivity (2016, positive): “As for question (ii), both classical [10, 9, 3] and more recent [40, 22] results from statistical learning theory show that as the number of examples grows in comparison to the size of the network the empirical loss must be close to the population loss.”
Application of Gaussian Process Regression Model to Predict Discharge Coefficient of Gated Piano Key Weir (2019, background): “As per such suggestions, approximately 70% of total data is sufficient for network training and the remaining 30% is used for network testing (Baum and Haussler 1989; Zare et al. 2012).”
The use of backpropagating artificial neural networks in land cover classification (2003, background): “Baum and Haussler ( 1989 ) propose that, if an accuracy level of 90% is desired, the number of training samples should be about 10 times the VC dimension, or the number of weights in the network.”
On the Use of the Adjusted Rand Index as a Metric for Evaluating Supervised Classification (2009, methodology): “For that purpose we took into account the minimum number of lines needed to separate the checkerboard classes and the well-known rule of thumb nh = w/ (based on a formula given in [12]), where w is the number of weights and the expected error rate.”
An Aggregation Approach to Short-Term Traffic Flow Prediction (2009, background): “Box and Jenkins [33] developed a practical approach to building ARIMA models that has had a fundamental impact on time series analysis and forecasting applications.”
A Simple Weight Decay Can Improve Generalization (1991, background): “Many recent studies have shown that the generalization ability of a neural network (or any other 'learning machine') depends on a balance between the information in the training examples and the complexity of the network, see for instance [1,2,3].”
Improving Generalization with Active Learning (1994, methodology): “Baum and Haussler (1989) examine the problem analytically for neural networks; Conn and Tesauro (1992) provide an empirical study of neural network generalization when learning from examples.”