When Good Equations Get Bad Scores: Improving Symbolic Regression Through Better Parameter Optimization

• URL: http://arxiv.org/abs/2605.23272v1
• Date: Fri, 22 May 2026 06:23:28 GMT
• Title: When Good Equations Get Bad Scores: Improving Symbolic Regression Through Better Parameter Optimization
• Abstract summary: Symbolic Regression (SR) plays a central role in scientific knowledge discovery by distilling mathematical equations from observational data.<n>We propose SAGE-Fit, an SR fitting framework that exploits the dual priors of symbolic expressions.
• Score: 24.34939435234426
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Symbolic Regression (SR) plays a central role in scientific knowledge discovery by distilling mathematical equations from observational data. Most existing SR methods function within a bi-level optimization framework: an outer loop that searches for the discrete equation structure, and an inner loop that optimizes the continuous parameters of that structure. Crucially, parameter-fitting quality directly determines a structure's score and thus the outer-loop search. However, nonlinear operators make the inner loop highly non-convex, and budget-driven reliance on fast local solvers (e.g., BFGS) often yields poor local minima and underestimated scores for correct structures. This ``Good Structure, Bad Score'' phenomenon becomes a key bottleneck, degrading efficiency and misguiding the search away from the true equation. To resolve this, we propose SAGE-Fit (Structure-Aware and Semantics-Guided Evaluator for Symbolic Regression), an SR-native fitting framework that exploits the dual native priors of symbolic expressions. By capitalizing on the structural and semantic priors unique to SR, we design tailored modules for each property, thereby effectively mitigating this optimization bottleneck. Extensive experiments demonstrate that our approach, as a plug-and-play module, significantly enhances evaluation fidelity and universally improves the performance of various SR systems.

Related papers

• FePySR: A Neural Feature Extraction Framework for Efficient and Scalable Symbolic Regression [4.327979408469824]
  We introduce FePySR, a two-stage framework that reduces the SR search space by extracting valid features prior to equation search.<n>Across five standard benchmarks, FePySR outperforms state-of-the-art methods by achieving higher equation recovery rates.<n>FePySR is a generalizable framework that can enhance the SR solvers, enabling the efficient and reliable recovery of symbolic expressions.
  arXiv  Detail & Related papers  (2026-05-12T20:04:59Z)
• Neural Structure Embedding for Symbolic Regression via Continuous Structure Search and Coefficient Optimization [8.496933324334167]
  SRCO is a unified embedding-driven framework for symbolic regression.<n>It transforms symbolic structures into a continuous, optimizable representation space.<n> Experiments on synthetic and real-world datasets show that our approach consistently outperforms state-of-the-art methods in equation accuracy, robustness, and search efficiency.
  arXiv  Detail & Related papers  (2026-03-23T18:04:03Z)
• Prior-Informed Neural Network Initialization: A Spectral Approach for Function Parameterizing Architectures [3.0098885383612104]
  We propose a prior-informed design strategy that leverages the intrinsic spectral and temporal structure of the data.<n>A principled methodology is introduced that uses the Fast Fourier Transform to extract dominant seasonal priors.<n>Experiments on synthetic and real-world benchmarks demonstrate that embedding data-driven priors significantly accelerates convergence.
  arXiv  Detail & Related papers  (2026-03-17T11:08:52Z)
• GENSR: Symbolic Regression Based in Equation Generative Space [15.186848349610363]
  GenSR is a generative latent space-based SR framework.<n>From a Bayesian perspective, GenSR reframes the SR task as maximizing the conditional distribution $p(mathrmEqu. mid mathrmNum.)$.
  arXiv  Detail & Related papers  (2026-02-24T05:14:34Z)
• Breaking the Simplification Bottleneck in Amortized Neural Symbolic Regression [8.351253396371686]
  SimpliPy is a rule-based simplification engine achieving a 100-fold speed-up over SymPy at comparable quality.<n>We demonstrate these advantages in our Flash-ANSR framework, which achieves much better accuracy than amortized baselines.
  arXiv  Detail & Related papers  (2026-02-09T16:47:00Z)
• Efficient Differentiable Causal Discovery via Reliable Super-Structure Learning [51.20606796019663]
  We propose ALVGL, a novel and general enhancement to the differentiable causal discovery pipeline.<n>ALVGL employs a sparse and low-rank decomposition to learn the precision matrix of the data.<n>We show that ALVGL not only achieves state-of-the-art accuracy but also significantly improves optimization efficiency.
  arXiv  Detail & Related papers  (2026-01-09T02:18:59Z)
• Unleashing Degradation-Carrying Features in Symmetric U-Net: Simpler and Stronger Baselines for All-in-One Image Restoration [52.82397287366076]
  All-in-one image restoration aims to handle diverse degradations (e.g., noise, blur, adverse weather) within a unified framework.<n>In this work, we reveal a critical insight: well-crafted feature extraction inherently encodes degradation-carrying information.<n>Our symmetric design preserves intrinsic degradation signals robustly, rendering simple additive fusion in skip connections.
  arXiv  Detail & Related papers  (2025-12-11T12:20:31Z)
• Stable Nonconvex-Nonconcave Training via Linear Interpolation [51.668052890249726]
  This paper presents a theoretical analysis of linearahead as a principled method for stabilizing (large-scale) neural network training. We argue that instabilities in the optimization process are often caused by the nonmonotonicity of the loss landscape and show how linear can help by leveraging the theory of nonexpansive operators.
  arXiv  Detail & Related papers  (2023-10-20T12:45:12Z)
• Performance Embeddings: A Similarity-based Approach to Automatic Performance Optimization [71.69092462147292]
  Performance embeddings enable knowledge transfer of performance tuning between applications. We demonstrate this transfer tuning approach on case studies in deep neural networks, dense and sparse linear algebra compositions, and numerical weather prediction stencils.
  arXiv  Detail & Related papers  (2023-03-14T15:51:35Z)
• Shapley-NAS: Discovering Operation Contribution for Neural Architecture Search [96.20505710087392]
  We propose a Shapley value based method to evaluate operation contribution (Shapley-NAS) for neural architecture search. We show that our method outperforms the state-of-the-art methods by a considerable margin with light search cost.
  arXiv  Detail & Related papers  (2022-06-20T14:41:49Z)
• Sparse PCA via $l_{2,p}$-Norm Regularization for Unsupervised Feature Selection [138.97647716793333]
  We propose a simple and efficient unsupervised feature selection method, by combining reconstruction error with $l_2,p$-norm regularization. We present an efficient optimization algorithm to solve the proposed unsupervised model, and analyse the convergence and computational complexity of the algorithm theoretically.
  arXiv  Detail & Related papers  (2020-12-29T04:08:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.