Learning Equations for Extrapolation and Control

By Subham S. Sahoo, Christoph H. Lampert, Georg Martius. International Conference on Machine Learning, 2018

306 citations recorded by the source corpus.

We present an approach to identify concise equations from data using a shallow neural network approach. In contrast to ordinary black-box regression, this approach allows understanding functional relations and generalizing them from observed data to unseen parts of the parameter space. We show how to extend the class of learnable equations for a recently proposed equation learning network to include divisions, and we improve the learning and model selection strategy to be useful for challenging real-world data. For systems governed by analytical expressions, our method can in many cases identify the true underlying equation and extrapolate to unseen domains. We demonstrate its effectiveness by experiments on a cart-pendulum system, where only 2 random rollouts are required to learn the forward dynamics and successfully achieve the swing-up task.

Subfields: Artificial Intelligence, Control and Systems Engineering, Statistical and Nonlinear Physics

View Paper: https://www.semanticscholar.org/paper/660f08f5aa355c19a851d848e0c1321c5c32c1ab

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Control and Systems Engineering

In Statistical and Nonlinear Physics

Citations in the corpus

Implicit Neural Representations with Periodic Activation Functions (2020, background): “Other work explores neural networks with periodic activations for simple classification tasks [21–23], equation learning [24], and recurrent neural networks [25–29].”
Interpretable scientific discovery with symbolic regression: a review (2022, unknown): “An extended version EQL ÷ [48] includes only the division, whereas exponential and logarithm activation functions are not included because of numerical issues.”
PDE-Net 2.0: Learning PDEs from Data with A Numeric-Symbolic Hybrid Deep Network (2018, methodology): “Our design of SymNetm is motivated by EQL/EQL ÷ proposed by [41, 42].”
Contemporary Symbolic Regression Methods and their Relative Performance (2021, background): “[21] Subham Sahoo, Christoph Lampert, and Georg Martius.”
Discovering Symbolic Models from Deep Learning with Inductive Biases (2020, methodology): “[27] is an interesting approach that uses gradient descent on a pre-defined equation up to some depth, parametrized with a neural network, instead of genetic algorithms; [33] uses gradient descent on a latent embedding of an equation; and [34] demonstrates Monte Carlo Tree Search as a symbolic regre”
Evaluating explorative prediction power of machine learning algorithms for materials discovery using k-fold forward cross-validation (2020, background): “Other related work on extrapolation of machine learning models is reported in [28,29].”
End-to-end symbolic regression with transformers (2022, unknown): “[21, 22] propose modiﬁcations to feed-forward networks to include interpretable components, i.e. replacing usual activation functions by operators such as cos , sin , however these are hard to optimize and prone to numerical issues.”