ParFam -- (Neural Guided) Symbolic Regression Based on Continuous Global Optimization

• URL: http://arxiv.org/abs/2310.05537v3
• Date: Wed, 29 May 2024 11:41:47 GMT
• Title: ParFam -- (Neural Guided) Symbolic Regression Based on Continuous Global Optimization
• Authors: Philipp Scholl, Katharina Bieker, Hillary Hauger, Gitta Kutyniok,
• Abstract summary: We present our new approach ParFam to translate the discrete symbolic regression problem into a continuous one. In combination with a global, this approach results in a highly effective method to tackle the problem of SR. We also present an extension incorporating a pre-trained transformer network DL-ParFam to guide ParFam.
• Score: 14.146976111782466
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: The problem of symbolic regression (SR) arises in many different applications, such as identifying physical laws or deriving mathematical equations describing the behavior of financial markets from given data. Various methods exist to address the problem of SR, often based on genetic programming. However, these methods are usually complicated and involve various hyperparameters. In this paper, we present our new approach ParFam that utilizes parametric families of suitable symbolic functions to translate the discrete symbolic regression problem into a continuous one, resulting in a more straightforward setup compared to current state-of-the-art methods. In combination with a global optimizer, this approach results in a highly effective method to tackle the problem of SR. We theoretically analyze the expressivity of ParFam and demonstrate its performance with extensive numerical experiments based on the common SR benchmark suit SRBench, showing that we achieve state-of-the-art results. Moreover, we present an extension incorporating a pre-trained transformer network DL-ParFam to guide ParFam, accelerating the optimization process by up to two magnitudes. Our code and results can be found at https://github.com/Philipp238/parfam.

Related papers

• Ab initio nonparametric variable selection for scalable Symbolic Regression with large $p$ [2.222138965069487]
  Symbolic regression (SR) is a powerful technique for discovering symbolic expressions that characterize nonlinear relationships in data. Existing SR methods do not scale to datasets with a large number of input variables, which are common in modern scientific applications. We propose PAN+SR, which combines ab initio nonparametric variable selection with SR to efficiently pre-screen large input spaces.
  arXiv  Detail & Related papers  (2024-10-17T15:41:06Z)
• A Functional Analysis Approach to Symbolic Regression [0.990319860068191]
  Symbolic regression (SR) poses a significant challenge for randomized searchs. Traditional genetic programming (GP) algorithms exhibit limited performance when tree-based representations are used for SR. We introduce a novel SR approach that draws insights from functional analysis.
  arXiv  Detail & Related papers  (2024-02-09T10:24:47Z)
• SymbolNet: Neural Symbolic Regression with Adaptive Dynamic Pruning [1.0356366043809717]
  We propose a neural network approach to symbolic regression in a novel framework that allows dynamic pruning of model weights, input features, and mathematical operators in a single training process. Our approach enables symbolic regression to achieve fast inference with nanosecond-scale latency on FPGAs for high-dimensional datasets in environments with stringent computational resource constraints.
  arXiv  Detail & Related papers  (2024-01-18T12:51:38Z)
• Deep Generative Symbolic Regression [83.04219479605801]
  Symbolic regression aims to discover concise closed-form mathematical equations from data. Existing methods, ranging from search to reinforcement learning, fail to scale with the number of input variables. We propose an instantiation of our framework, Deep Generative Symbolic Regression.
  arXiv  Detail & Related papers  (2023-12-30T17:05:31Z)
• End-to-End Meta-Bayesian Optimisation with Transformer Neural Processes [52.818579746354665]
  This paper proposes the first end-to-end differentiable meta-BO framework that generalises neural processes to learn acquisition functions via transformer architectures. We enable this end-to-end framework with reinforcement learning (RL) to tackle the lack of labelled acquisition data.
  arXiv  Detail & Related papers  (2023-05-25T10:58:46Z)
• Differentiable Genetic Programming for High-dimensional Symbolic Regression [13.230237932229052]
  Symbolic regression (SR) is considered an effective way to reach interpretable machine learning (ML) Genetic programming (GP) has been the dominator in solving SR problems. We propose a differentiable approach named DGP to construct GP trees towards high-dimensional SR.
  arXiv  Detail & Related papers  (2023-04-18T11:39:45Z)
• Relational Reasoning via Set Transformers: Provable Efficiency and Applications to MARL [154.13105285663656]
  A cooperative Multi-A gent R einforcement Learning (MARL) with permutation invariant agents framework has achieved tremendous empirical successes in real-world applications. Unfortunately, the theoretical understanding of this MARL problem is lacking due to the curse of many agents and the limited exploration of the relational reasoning in existing works. We prove that the suboptimality gaps of the model-free and model-based algorithms are independent of and logarithmic in the number of agents respectively, which mitigates the curse of many agents.
  arXiv  Detail & Related papers  (2022-09-20T16:42:59Z)
• Sparse high-dimensional linear regression with a partitioned empirical Bayes ECM algorithm [62.997667081978825]
  We propose a computationally efficient and powerful Bayesian approach for sparse high-dimensional linear regression. Minimal prior assumptions on the parameters are used through the use of plug-in empirical Bayes estimates. The proposed approach is implemented in the R package probe.
  arXiv  Detail & Related papers  (2022-09-16T19:15:50Z)
• GSR: A Generalized Symbolic Regression Approach [13.606672419862047]
  Generalized Symbolic Regression presented in this paper. We show that our GSR method outperforms several state-of-the-art methods on the well-known Symbolic Regression benchmark problem sets. We highlight the strengths of GSR by introducing SymSet, a new SR benchmark set which is more challenging relative to the existing benchmarks.
  arXiv  Detail & Related papers  (2022-05-31T07:20:17Z)
• A general sample complexity analysis of vanilla policy gradient [101.16957584135767]
  Policy gradient (PG) is one of the most popular reinforcement learning (RL) problems. "vanilla" theoretical understanding of PG trajectory is one of the most popular methods for solving RL problems.
  arXiv  Detail & Related papers  (2021-07-23T19:38:17Z)
• Convergence of Meta-Learning with Task-Specific Adaptation over Partial Parameters [152.03852111442114]
  Although model-agnostic metalearning (MAML) is a very successful algorithm meta-learning practice, it can have high computational complexity. Our paper shows that such complexity can significantly affect the overall convergence performance of ANIL.
  arXiv  Detail & Related papers  (2020-06-16T19:57:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.