Fisher information and stochastic complexity

By Jorma Rissanen. 1996

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Overcoming Multi-Model Forgetting (2019, positive): “Kirk-patrick et al. (2017) selectively slow down the learning of the weights that are comparatively important for the ﬁrst task by deﬁning the importance of an individual weight us-ing its Fisher information (Rissanen, 1996).”
Mining product reputations on the Web (2002, methodology): “In constructing classification and association rules, we employ stochastic complexity [21] as a criterion for selecting words.”
The power of quantum neural networks (2020, background): “In particular, measures such as the effective dimension have been motivated from an information-theoretic standpoint and depend on the Fisher information; a quantity that describes the geometry of a model’s parameter space and is essential in both statistics and machine learning [20–22].”
Optimal Model Assessment, Selection, and Combination (2006, background): “This implies that it is optimal among all other competitors in FMS, such as AIC, BIC, RIC, CIC, Rissanen’s modified stochastic complexity (Rissanen 1996), and CV, to name a few.”
Between the Devil and the Deep Blue Sea: Tensions Between Scientific Judgement and Statistical Model Selection (2018, background): “For instance,97 in one of my early papers (Navarro 2004) I documented an issue with (a specific version98 of) the minimum description length criterion developed by Rissanen (1996) and introduced99 to psychology by Pitt, Myung and Zhang (2002).”
A survey of Bayesian Network structure learning (2021, methodology): “The BIC score is widely popular and has been found to be able to learn the true network faster than other scoring functions such as AIC, BDeu and fNML (Liu et al., 2012).”
The Minimum Description Length Principle in Coding and Modeling (1998, methodology): “To bolster the connection given here between the individual risk of efficient estimators of order and the optimal cumulative risk or redundancy of order , we mention here that classic results on negligibility of the set of parameter values for which an estimator is superefficient (LeCam [35] assumin”
A tutorial introduction to the minimum description length principle (2004, methodology): “This development culminated in 1996 in a completely precise prescription of MDL for many, but certainly not all practical situations [Rissanen 1996].”
Predictability , Complexity , and Learning (2002, background): “…needs to encode a data string (minimum description length, or MDL) within a certain assumed model class was termed by Rissanen stochastic complexity, and in recent work he refers to the piece of the stochastic complexity required for coding the parameters as the model complexity (Rissanen, 1996).”
Predictability, Complexity, and Learning (2000, background): “…one needs to encode a data string (minimum description length or MDL) within a certain assumed model class was termed by Rissanen stochastic complexity, and in recent work he refers to the piece of the stochastic complexity required for coding the parameters as the model complexity (Rissanen 1996).”
An efficient way to learn rules for grapheme-to-phoneme conversion in Chinese (2002, unknown): “The amount of information is measured in term of Stochastic Complexity (SC)[8].”