Distilling Free-Form Natural Laws from Experimental Data
By Michael D. Schmidt, Hod Lipson. 2009
3,110 citations recorded by the source corpus.
For centuries, scientists have attempted to identify and document analytical laws that underlie physical phenomena in nature. Despite the prevalence of computing power, the process of finding natural laws and their corresponding equations has resisted automation. A key challenge to finding analytic relations automatically is defining algorithmically what makes a correlation in observed data important and insightful. We propose a principle for the identification of nontriviality. We demonstrated this approach by automatically searching motion-tracking data captured from various physical systems, ranging from simple harmonic oscillators to chaotic double-pendula. Without any prior knowledge about physics, kinematics, or geometry, the algorithm discovered Hamiltonians, Lagrangians, and other laws of geometric and momentum conservation. The discovery rate accelerated as laws found for simpler systems were used to bootstrap explanations for more complex systems, gradually uncovering the “alphabet” used to describe those systems.
Subfields: Artificial Intelligence, Numerical Methods & Scientific Computing, AI for Physical Sciences, Signal Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yoshua Bengio
- Mohit Bansal
- Sergey Levine
- Jianfeng Gao
- Quoc V. Le
- Percy Liang
- Josh Tenenbaum
- Ruslan Salakhutdinov
In Numerical Methods & Scientific Computing
- Maizar Raissi
- G. Em Karniadakis
- Steven L. Brunton
- Paris Perdikaris
- J. Nathan Kutz
- Dongkun Zhang
- Joshua L. Proctor
- Liu Yang
In AI for Physical Sciences
- G. Em Karniadakis
- Maizar Raissi
- Steven L. Brunton
- Paris Perdikaris
- J. Nathan Kutz
- Lu Lu
- Liu Yang
- Dongkun Zhang
In Signal Processing
- Steven L. Brunton
- J. Nathan Kutz
- Joshua L. Proctor
- Zoubin Ghahramani
- Maizar Raissi
- Georg Martius
- Christoph H. Lampert
- G. Em Karniadakis
Citations in the corpus
- Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations (2018, background): “, 2011; Daniels and Nemenman, 2015a,b), and symbolic regression (Bongard and Lipson, 2007; Schmidt and Lipson, 2009).”
- Multistep Neural Networks for Data-driven Discovery of Nonlinear Dynamical Systems (2018, background): “More recently, approaches based on symbolic regression [15], sparse regression, and compressive sensing [16, 17] were able to go beyond estimating a black-box approximation of the dynamics given by f , and return more interpretable models that can uncover the full parametric form of an underlying…”
- Data-driven discovery of partial differential equations (2016, methodology): “Both the evolutionary (14) and sparse (16) symbolic regression methods avoid overfitting by selecting parsimonious models that balance model accuracy with complexity via Pareto analysis.”
- Discovering governing equations from data by sparse identification of nonlinear dynamical systems (2015, background): “A seminal breakthrough by Bongard and Lipson (3) and Schmidt and Lipson (4) has resulted in a new approach to determine the underlying structure of a nonlinear dynamical system from data.”
- Grammar Variational Autoencoder (2017, background): “Learning arithmetic expressions to fit data, often called symbolic regression , are generally based on genetic programming (Willis et al., 1997) or other computationally demanding evolutionary algorithms to propose candidate expressions (Schmidt & Lipson, 2009).”
- Hidden physics models: Machine learning of nonlinear partial differential equations (2017, background): “The literature on data-driven discovery of dynamical systems [3] is vast and encompasses equation-free modeling [4], artificial neural networks [5], nonlinear regression [6], empirical dynamic modeling [7,8], modeling emergent behavior [9], automated inference of dynamics [10–12], normal form identi”
- Automatic Construction and Natural-Language Description of Nonparametric Regression Models (2014, background): “Equation learning Todorovski and Dzeroski (1997), Washio et al. (1999) and Schmidt and Lipson (2009) learn parametric forms of functions specifying time series, or relations between quantities.”
- Efficient Inference of Parsimonious Phenomenological Models of Cellular Dynamics Using S-Systems and Alternating Regression (2014, background): “The model of oscillations in yeast glycolysis [15] has become a standard test case for biochemical dynamics inference [3, 10].”
- PDE-Net: Learning PDEs from Data (2017, background): “One of earlier attempts on data-driven discovery of hidden physical laws is by Bongard & Lipson (2007) and Schmidt & Lipson (2009).”
- Learning Equations for Extrapolation and Control (2018, methodology): “We also compare to Eureqa (Nutonian, 2018), a symbolic regression algorithm using evolutionary search introduced in (Schmidt & Lipson, 2009).”
- Virtual screening: an endless staircase? (2010, background): “A first step in this direction has recently been made by Schmidt and Lipson, who presented a computational concept for devising 'free-form natural laws' from experimental dat”
- Data-driven Discovery of Partial Differential Equations for Multiple-Physics Electromagnetic Problem (2019, background): “One of earlier research on data-driven discovery for free-form natural laws is proposed by Bongard and Lipson (2007) and Schmidt and Lipson (2009) [1].”
- Quality and Diversity Optimization: A Unifying Modular Framework (2017, methodology): “For example, evolutionary methods have been used to optimize the morphologies and the neural networks of physical robots [7], and to infer the equations behind collected data [8].”
- Integrating Scientific Knowledge with Machine Learning for Engineering and Environmental Systems (2020, background): “[38, 244], where they apply symbolic regression to differences between computed derivatives and analytic derivatives to determine underlying dynamical systems.”
- Systems biology informed deep learning for inferring parameters and hidden dynamics (2019, background): “The model of oscillations in yeast glycolysis [33] has become a standard benchmark 160 problem for systems biology inference [34,35] as it represents complex nonlinear 161 dynamics typical of biological systems.”