Neural Symbolic Regression that Scales

• URL: http://arxiv.org/abs/2106.06427v1
• Date: Fri, 11 Jun 2021 14:35:22 GMT
• Title: Neural Symbolic Regression that Scales
• Authors: Luca Biggio, Tommaso Bendinelli, Alexander Neitz, Aurelien Lucchi, Giambattista Parascandolo
• Abstract summary: We introduce the first symbolic regression method that leverages large scale pre-training. We procedurally generate an unbounded set of equations, and simultaneously pre-train a Transformer to predict the symbolic equation from a corresponding set of input-output-pairs.
• Score: 58.45115548924735
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Symbolic equations are at the core of scientific discovery. The task of discovering the underlying equation from a set of input-output pairs is called symbolic regression. Traditionally, symbolic regression methods use hand-designed strategies that do not improve with experience. In this paper, we introduce the first symbolic regression method that leverages large scale pre-training. We procedurally generate an unbounded set of equations, and simultaneously pre-train a Transformer to predict the symbolic equation from a corresponding set of input-output-pairs. At test time, we query the model on a new set of points and use its output to guide the search for the equation. We show empirically that this approach can re-discover a set of well-known physical equations, and that it improves over time with more data and compute.

Related papers

• A Comparison of Recent Algorithms for Symbolic Regression to Genetic Programming [0.0]
  Symbolic regression aims to model and map data in a way that can be understood by scientists. Recent advancements, have attempted to bridge the gap between these two fields.
  arXiv  Detail & Related papers  (2024-06-05T19:01:43Z)
• Deep Generative Symbolic Regression [83.04219479605801]
  Symbolic regression aims to discover concise closed-form mathematical equations from data. Existing methods, ranging from search to reinforcement learning, fail to scale with the number of input variables. We propose an instantiation of our framework, Deep Generative Symbolic Regression.
  arXiv  Detail & Related papers  (2023-12-30T17:05:31Z)
• Predicting Ordinary Differential Equations with Transformers [65.07437364102931]
  We develop a transformer-based sequence-to-sequence model that recovers scalar ordinary differential equations (ODEs) in symbolic form from irregularly sampled and noisy observations of a single solution trajectory. Our method is efficiently scalable: after one-time pretraining on a large set of ODEs, we can infer the governing law of a new observed solution in a few forward passes of the model.
  arXiv  Detail & Related papers  (2023-07-24T08:46:12Z)
• ResMem: Learn what you can and memorize the rest [79.19649788662511]
  We propose the residual-memorization (ResMem) algorithm to augment an existing prediction model. By construction, ResMem can explicitly memorize the training labels. We show that ResMem consistently improves the test set generalization of the original prediction model.
  arXiv  Detail & Related papers  (2023-02-03T07:12:55Z)
• Discovering ordinary differential equations that govern time-series [65.07437364102931]
  We propose a transformer-based sequence-to-sequence model that recovers scalar autonomous ordinary differential equations (ODEs) in symbolic form from time-series data of a single observed solution of the ODE. Our method is efficiently scalable: after one-time pretraining on a large set of ODEs, we can infer the governing laws of a new observed solution in a few forward passes of the model.
  arXiv  Detail & Related papers  (2022-11-05T07:07:58Z)
• SymbolicGPT: A Generative Transformer Model for Symbolic Regression [3.685455441300801]
  We present SymbolicGPT, a novel transformer-based language model for symbolic regression. We show that our model performs strongly compared to competing models with respect to the accuracy, running time, and data efficiency.
  arXiv  Detail & Related papers  (2021-06-27T03:26:35Z)
• Shape-constrained Symbolic Regression -- Improving Extrapolation with Prior Knowledge [0.0]
  The aim is to find models which conform to expected behaviour and which have improved capabilities. The algorithms are tested on a set of 19 synthetic and four real-world regression problems. Shape-constrained regression produces the best results for the test set but also significantly larger models.
  arXiv  Detail & Related papers  (2021-03-29T14:04:18Z)
• AI Feynman 2.0: Pareto-optimal symbolic regression exploiting graph modularity [8.594811303203581]
  We present an improved method for symbolic regression that seeks to fit data to formulas that are Pareto-optimal. It improves on the previous state-of-the-art by typically being orders of magnitude more robust toward noise and bad data. We develop a method for discovering generalized symmetries from gradient properties of a neural network fit.
  arXiv  Detail & Related papers  (2020-06-18T18:01:19Z)
• The data-driven physical-based equations discovery using evolutionary approach [77.34726150561087]
  We describe the algorithm for the mathematical equations discovery from the given observations data. The algorithm combines genetic programming with the sparse regression. It could be used for governing analytical equation discovery as well as for partial differential equations (PDE) discovery.
  arXiv  Detail & Related papers  (2020-04-03T17:21:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.