Information, Physics, and Computation

By M. M'ezard, Andrea Montanari. 2009

339 citations recorded by the source corpus.

Subfields: Computational Theory and Mathematics

View Paper: https://www.semanticscholar.org/paper/d2b88df6f4987ee2b34d196d9310481c4cf277be

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Theory and Mathematics

Citations in the corpus

Constraint satisfaction problems and neural networks: A statistical physics perspective. (2008, background): “Thermodynamical quantities such as the free-energy log Z can also be derived (Mézard and Montanari, 2009) from the messages ðgi!a;ga!iÞ.”
Structured Learning and Prediction in Computer Vision (2011, background): “[171] showed a close connection between the so called Bethe free energy approximation in physics and the loopy belief propagation algorithm [104].”
Message passing algorithms for compressed sensing: I. motivation and construction (2009, background): “n describe the output of the corresponding AMP, when the latter is run for a sufﬁciently large number of iterations (independent of the dimensions n,N). It is well known, within statistical mechanics [9], that the ﬁxed point equations do indeed coincide with the equations obtained through a complete”
Statistical mechanics of complex neural systems and high dimensional data (2013, methodology): “corresponds to computing the marginal distribution of a single neuron in a full joint distribution given by (2). Here we introduce eﬃcient distributed message passing algorithms from computer science [5, 11, 10] that have been developed to compute such marginals in probability distributions which ob”
Vital nodes identification in complex networks (2016, background): “n a dynamics that can be represented by a static constraint-satisfaction model over discrete variables. Therefore, the message-passing theory has found wide applications in combinational optimization [252] and becomes a popular tool in network analysis [253]. This section will introduce two examples”
A high-bias, low-variance introduction to Machine Learning for physicists (2018, neutral): “This is a rich area of activity at the intersection of physics, computer science, information theory, and machine learning and interested readers are encouraged to consult the literature for further information (see also (Mezard and Montanari, 2009)).”
Entropic gradient descent algorithms and wide flat minima (2020, methodology): “On relatively tractable shallow networks, the local entropy of any given conﬁguration can be computed efﬁciently using Belief Propagation, and it can be also used directly as a training objective.”
Graphical Models, Exponential Families, and Variational Inference (2008, background): “Random constructions of factor graphs on m vertices with bounded degree have cycles of typical length ≍ logm; this tree-like property is essential to the success of the sum-product algorithm for approximate decoding [196, 166].”
Advanced Structured Prediction (2014, methodology): “This conservative language is known in statistical mechanics as the Potts model with external field (Mezard and Montanari, 2009) and is frequently used for image segmentation (Rother et al.”
Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Loss (2020, background): “(7) When ` is the exponential, this function is known as the soft-min or the free energy and is concave (Mézard and Montanari, 2009).”
Extreme value statistics of correlated random variables: A pedagogical review (2019, background): “Recently, there has been a surge of interest in these questions in the context of deep learning and artificial intelligence [276–279].”
Probabilistic reconstruction in compressed sensing: algorithms, phase diagrams, and threshold achieving matrices (2012, methodology): “To approximate the expectations we will use a variant of the belief propagation (BP) algorithm [28, 29, 46].”
Identification of direct residue contacts in protein–protein interaction by message passing (2009, background): “In recent years, insights from spin-glass physics have led to the development of generalized message-passing techniques, which have been applied successfully to a number of hard combinatorial problems such as K-SAT (19-21).”