Occam's razor

By Temple F. Smith. 1980

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Philosophy

Citations in the corpus

True Few-Shot Learning with Language Models (2021, background): “MDL measures generalization because it evaluates how much a learning algorithm compresses the labels y 1: N given the inputs x 1: N , and because better compression implies better generalization [33].”
The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks (2018, methodology): “We observe a pattern of increasing and then decreasing test accuracy as we prune, forming an Occam’s Hill (Rasmussen & Ghahramani, 2001) where the original, overparameterized model has too much complexity (perhaps overfitting) and the extremely pruned model has too little.”
Deep learning in neural networks: An overview (2014, methodology): “Given some programming language, the principle of Minimum Description Length (MDL) can be used to measure the complexity of a solution candidate by the length of the shortest program that computes it (e.g., Solomonoff, 1964; Kolmogorov, 1965b; Chaitin, 1966; Wallace and Boulton, 1968; Levin, 1973a; ”
Deep Kernel Learning (2015, methodology): “(4) pleasingly separates into automatically calibrated model ﬁt and complexity terms (Rasmussen and Ghahramani, 2001).”
Stochastic Variational Deep Kernel Learning (2016, background): “Gaussian processes are promising for large data problems, because they can grow their information capacity with the amount of available data, in combination with automatically calibrated model complexity [21, 25].”
Hidden physics models: Machine learning of nonlinear partial differential equations (2017, background): “This regularization mechanism automatically meets the Occam’s razor principle [52] which encourages simplicity in explanations.”
Kernel Interpolation for Scalable Structured Gaussian Processes (KISS-GP) (2015, methodology): “Although we cannot eﬃciently eigendecompose K ( m ) + A , we can take matrix vector products ( K n ) to within machine precision, and perform eﬃcient inference, using iterative methods such as linear conjugate gradients, which only involve matrix vector products.”
Automatic Construction and Natural-Language Description of Nonparametric Regression Models (2014, methodology): “BIC trades off model ﬁt and complexity and implements what is known as “Bayesian Occam’s Razor” (e.g. Rasmussen and Ghahramani, 2001; MacKay, 2003).”
Bayesian Deep Learning and a Probabilistic Perspective of Generalization (2020, background): “Rasmussen and Ghahramani [41] reasons about how the marginal likelihood can favour large flexible models, as long as they correspond to a reasonable distribution over functions.”
Numerical Gaussian Processes for Time-Dependent and Nonlinear Partial Differential Equations (2017, background): “This effect is known as Occam’s razor [27] after William of Occam (1285–1349), who encouraged simplicity in explanations by the principle: “Plurality should not be assumed without necessity.”
Machine learning of linear differential equations using Gaussian processes (2017, background): “This effect is called Occam’s razor [40] after William of Occam (1285–1349), who encouraged simplicity in explanations by the principle: “plurality should not be assumed without necessity”.”
Latent-Space Variational Bayes (2008, unknown): “In particular, the probability P ðY jMÞ, called the marginal likelihood or model evidence, has been shown to penalize overcomplex models by automatically encoding Occam's Razor [1], [2], so it is important for model comparison [3].”
Automatic model construction with Gaussian processes (2014, methodology): “We choose marginal likelihood as our criterion, since it balances the fit and complexity of a model (Rasmussen and Ghahramani, 2001).”
Features in Concert: Discriminative Feature Selection meets Unsupervised Clustering (2014, background): “They seem to follow closer the Occam’s Razor principle [2], which would explain in part their good performance and generalization.”
Infinite latent feature models and the Indian buffet process (2005, background): “An alternative is to assume that the true dimension ality is unbounded, and that the observed objects manifest only a finite subset of classes or f eatures [5].”