The simplest model of jamming

By Silvio Franz, Giorgio Parisi. 2015

138 citations recorded by the source corpus.

We study a well known neural network model—the perceptron—as a simple statistical physics model of jamming of hard objects. We exhibit two regimes: (1) a convex optimization regime where jamming is hypostatic and non-critical; (2) a non-convex optimization regime where jamming is isostatic and critical. We characterize the critical jamming phase through exponents describing the distribution laws of forces and gaps. Surprisingly we find that these exponents coincide with the corresponding ones recently computed in high dimensional hard spheres. In addition, modifying the perceptron to a random linear programming problem, we show that isostaticity is not a sufficient condition for singular force and gap distributions. For that, fragmentation of the space of solutions (replica symmetry breaking) appears to be a crucial ingredient. We hypothesize universality for a large class of non-convex constrained satisfaction problems with continuous variables.

Subfields: Computational Theory and Mathematics, Computer Networks and Communications, Mathematical Physics, AI for Physical Sciences

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Theory and Mathematics

In Computer Networks and Communications

In Mathematical Physics

In AI for Physical Sciences

Citations in the corpus

Scaling description of generalization with number of parameters in deep learning (2019, background): “In the perceptron, the simplest network without hidden layers, the cusp in the test error at the jamming point is also observed and predicted analytically [50, 51, 52, 53, 54, 55].”
The jamming transition as a paradigm to understand the loss landscape of deep neural networks (2018, methodology): “[34] Silvio Franz and Giorgio Parisi, “The simplest model of”
A jamming transition from under- to over-parametrization affects generalization in deep learning (2018, background): “[41] Silvio Franz and Giorgio Parisi.”
Universality of the SAT-UNSAT (jamming) threshold in non-convex continuous constraint satisfaction problems (2017, methodology): “(83) is physically very important: in fact, the relation γ = 1 / ( 2 + θ ) has been proven in [ 7 ] to be a direct consequence of marginal stability (in the perceptron, the same marginal stability argument is discussed in [ 14 ] ).”
A jamming transition from under- to over-parametrization affects loss landscape and generalization (2018, background): “The manuscript [35], which appeared at the same time than ours, shows that the critical properties of the jamming transition found for the non-convex perceptron [14] hold more generally in some shallow networks.”
Universal spectrum of normal modes in low-temperature glasses (2015, background): “The spherical perceptron is probably the simplest abstract CCSP where, appropriately rephrased, the three ingredients above are combined (25).”
Triple descent and the two kinds of overfitting: where and why do they appear? (2020, unknown): “This peak1 was shown to be related (1)Also called the jamming peak due to similarities with a well-studied phenomenon in the Statistical Physics literature [14, 15, 16, 17, 18].”