Evolving Form and Function: Dual-Objective Optimization in Neural Symbolic Regression Networks

• URL: http://arxiv.org/abs/2502.17393v1
• Date: Mon, 24 Feb 2025 18:20:41 GMT
• Title: Evolving Form and Function: Dual-Objective Optimization in Neural Symbolic Regression Networks
• Authors: Amanda Bertschinger, James Bagrow, Joshua Bongard,
• Abstract summary: We introduce a method that combines gradient descent and evolutionary computation to yield neural networks that minimize the symbolic and behavioral errors of the equations they generate from data.<n>These evolved networks are shown to generate more symbolically and behaviorally accurate equations than those generated by networks trained by state-of-the-art gradient based neural symbolic regression methods.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Data increasingly abounds, but distilling their underlying relationships down to something interpretable remains challenging. One approach is genetic programming, which `symbolically regresses' a data set down into an equation. However, symbolic regression (SR) faces the issue of requiring training from scratch for each new dataset. To generalize across all datasets, deep learning techniques have been applied to SR. These networks, however, are only able to be trained using a symbolic objective: NN-generated and target equations are symbolically compared. But this does not consider the predictive power of these equations, which could be measured by a behavioral objective that compares the generated equation's predictions to actual data. Here we introduce a method that combines gradient descent and evolutionary computation to yield neural networks that minimize the symbolic and behavioral errors of the equations they generate from data. As a result, these evolved networks are shown to generate more symbolically and behaviorally accurate equations than those generated by networks trained by state-of-the-art gradient based neural symbolic regression methods. We hope this method suggests that evolutionary algorithms, combined with gradient descent, can improve SR results by yielding equations with more accurate form and function.

Related papers

• Neuro-Evolutionary Approach to Physics-Aware Symbolic Regression [0.0]
  We propose a neuro-evolutionary symbolic regression method that combines evolutionary-based search for optimal neural network topologies with gradient-based tuning of the network's parameters. Our method employs a memory-based strategy and population perturbations to enhance exploitation and reduce the risk of being trapped in suboptimal NNs.
  arXiv  Detail & Related papers  (2025-04-23T08:29:53Z)
• A Comparison of Recent Algorithms for Symbolic Regression to Genetic Programming [0.0]
  Symbolic regression aims to model and map data in a way that can be understood by scientists. Recent advancements, have attempted to bridge the gap between these two fields.
  arXiv  Detail & Related papers  (2024-06-05T19:01:43Z)
• SymbolNet: Neural Symbolic Regression with Adaptive Dynamic Pruning for Compression [1.0356366043809717]
  We propose $ttSymbolNet$, a neural network approach to symbolic regression specifically designed as a model compression technique.<n>This framework allows dynamic pruning of model weights, input features, and mathematical operators in a single training process.
  arXiv  Detail & Related papers  (2024-01-18T12:51:38Z)
• Deep Generative Symbolic Regression [83.04219479605801]
  Symbolic regression aims to discover concise closed-form mathematical equations from data. Existing methods, ranging from search to reinforcement learning, fail to scale with the number of input variables. We propose an instantiation of our framework, Deep Generative Symbolic Regression.
  arXiv  Detail & Related papers  (2023-12-30T17:05:31Z)
• Toward Physically Plausible Data-Driven Models: A Novel Neural Network Approach to Symbolic Regression [2.7071541526963805]
  This paper proposes a novel neural network-based symbolic regression method. It constructs physically plausible models based on even very small training data sets and prior knowledge about the system. We experimentally evaluate the approach on four test systems: the TurtleBot 2 mobile robot, the magnetic manipulation system, the equivalent resistance of two resistors in parallel, and the longitudinal force of the anti-lock braking system.
  arXiv  Detail & Related papers  (2023-02-01T22:05:04Z)
• A predictive physics-aware hybrid reduced order model for reacting flows [65.73506571113623]
  A new hybrid predictive Reduced Order Model (ROM) is proposed to solve reacting flow problems. The number of degrees of freedom is reduced from thousands of temporal points to a few POD modes with their corresponding temporal coefficients. Two different deep learning architectures have been tested to predict the temporal coefficients.
  arXiv  Detail & Related papers  (2023-01-24T08:39:20Z)
• Accelerating Understanding of Scientific Experiments with End to End Symbolic Regression [12.008215939224382]
  We develop a deep neural network to address the problem of learning free-form symbolic expressions from raw data. We train our neural network on a synthetic dataset consisting of data tables of varying length and varying levels of noise. We validate our technique by running on a public dataset from behavioral science.
  arXiv  Detail & Related papers  (2021-12-07T22:28:53Z)
• Training Feedback Spiking Neural Networks by Implicit Differentiation on the Equilibrium State [66.2457134675891]
  Spiking neural networks (SNNs) are brain-inspired models that enable energy-efficient implementation on neuromorphic hardware. Most existing methods imitate the backpropagation framework and feedforward architectures for artificial neural networks. We propose a novel training method that does not rely on the exact reverse of the forward computation.
  arXiv  Detail & Related papers  (2021-09-29T07:46:54Z)
• Neural Symbolic Regression that Scales [58.45115548924735]
  We introduce the first symbolic regression method that leverages large scale pre-training. We procedurally generate an unbounded set of equations, and simultaneously pre-train a Transformer to predict the symbolic equation from a corresponding set of input-output-pairs.
  arXiv  Detail & Related papers  (2021-06-11T14:35:22Z)
• A Bayesian Perspective on Training Speed and Model Selection [51.15664724311443]
  We show that a measure of a model's training speed can be used to estimate its marginal likelihood. We verify our results in model selection tasks for linear models and for the infinite-width limit of deep neural networks. Our results suggest a promising new direction towards explaining why neural networks trained with gradient descent are biased towards functions that generalize well.
  arXiv  Detail & Related papers  (2020-10-27T17:56:14Z)
• The data-driven physical-based equations discovery using evolutionary approach [77.34726150561087]
  We describe the algorithm for the mathematical equations discovery from the given observations data. The algorithm combines genetic programming with the sparse regression. It could be used for governing analytical equation discovery as well as for partial differential equations (PDE) discovery.
  arXiv  Detail & Related papers  (2020-04-03T17:21:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.