Symbolic Regression via Latent Iterative Refinement

• URL: http://arxiv.org/abs/2605.27245v1
• Date: Tue, 26 May 2026 16:25:05 GMT
• Title: Symbolic Regression via Latent Iterative Refinement
• Abstract summary: We propose Latent Equation Embedding (LEE), a framework that closes the gap between a one-shot prediction and the true posterior.<n>LEE learns a shared latent space Z equipped with three components.<n>LEE produces expressions 2--10x simpler than the strongest accuracy-oriented baselines.
• Score: 8.016092717336457
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Symbolic regression (SR) seeks closed-form mathematical expressions that fit observed data. Neural SR methods amortize the search by training an encoder to map observations directly to expressions in a single pass, but this amortized inference leaves a residual amortization gap between its one-shot prediction and the true posterior. We propose Latent Equation Embedding (LEE), a framework that closes this gap through iterative amortized inference in a functionally grounded latent space. LEE learns a shared latent space Z equipped with three components: an encoder f_theta that jointly embeds symbolic tokens and numerical observations into a single latent vector z; an expression decoder g_expr that reconstructs formulas from z; and an evaluation decoder g_eval that predicts function values from z, explicitly grounding the latent space in functional behavior. At inference, LEE performs iterative refinement by re-encoding decoded expressions jointly with observations, progressively improving the latent estimate. LEE uses the encoder itself as a learned inference optimizer: each re-encoding step implicitly computes the mismatch between the candidate and the data. Because g_eval is differentiable in z, we additionally interleave continuous gradient descent with discrete re-encoding, yielding a hybrid iterative and gradient refinement procedure. On SRBench across three noise levels, against 19 baselines spanning genetic programming, symbolic-neural hybrids, and pre-trained Transformers, LEE produces expressions 2--10x simpler than the strongest accuracy-oriented baselines, including Operon, GP-GOMEA, TPSR, RAG-SR, and GenSR, with complexity 8--11 versus 20--90. These results advance the low-complexity region of the accuracy-complexity Pareto frontier and show graceful degradation as noise increases.

Related papers

• SMILE: Bridging Continuous Optimization and Discrete Symbolic Recovery [29.456350389050574]
  Symbolic regression discovers closed-form mathematical expressions from data, offering interpretability beyond black-box models.<n>We introduce SMILE, a hybrid framework that unifies continuous gradient-based optimization with discrete symbolic recovery.<n>We evaluate SMILE on SRBench across ground-truth and black-box datasets, with ablation studies validating each component.
  arXiv  Detail & Related papers  (2026-09-04T02:20:00Z)
• InsightSR: Refining Symbolic Regression Search Spaces via Parallel Semantic and Structural LLM Guidance [8.272456038469484]
  Symbolic regression seeks to discover parsimonious mathematical laws from observational data.<n>We present InsightSR, a framework that embeds Large Language Models (LLMs) iteration as a guiding layer around the PySR genetic programming engine.
  arXiv  Detail & Related papers  (2026-08-26T02:04:20Z)
• Neural Structure Embedding for Symbolic Regression via Continuous Structure Search and Coefficient Optimization [8.496933324334167]
  SRCO is a unified embedding-driven framework for symbolic regression.<n>It transforms symbolic structures into a continuous, optimizable representation space.<n> Experiments on synthetic and real-world datasets show that our approach consistently outperforms state-of-the-art methods in equation accuracy, robustness, and search efficiency.
  arXiv  Detail & Related papers  (2026-03-23T18:04:03Z)
• In-Context Symbolic Regression for Robustness-Improved Kolmogorov-Arnold Networks [3.7163623763519733]
  Symbolic regression aims to replace blackbox predictors with analytical expressions that can be inspected and validated in machine learning.<n> Kolmogorov-Arnold Pursuit Networks (KANs) are well suited to this goal because each connection between adjacent units (an "edge") ismetd by a univariate function.<n>In practice, however, symbolic extraction is a bottleneck: the standard KAN-to-symbol approach fits operators to each learned edge function in fine isolation.
  arXiv  Detail & Related papers  (2026-03-16T13:21:26Z)
• GENSR: Symbolic Regression Based in Equation Generative Space [15.186848349610363]
  GenSR is a generative latent space-based SR framework.<n>From a Bayesian perspective, GenSR reframes the SR task as maximizing the conditional distribution $p(mathrmEqu. mid mathrmNum.)$.
  arXiv  Detail & Related papers  (2026-02-24T05:14:34Z)
• Unbiased Gradient Estimation for Event Binning via Functional Backpropagation [64.88399635309918]
  We propose a novel framework for unbiased gradient estimation of arbitrary binning functions by synthesizing weak derivatives during backpropagation.<n>We achieve 9.4% lower EPE in self-supervised optical flow, and 5.1% lower RMS error in SLAM, demonstrating broad benefits for event-based visual perception.
  arXiv  Detail & Related papers  (2026-02-13T04:05:03Z)
• Gaussian Match-and-Copy: A Minimalist Benchmark for Studying Transformer Induction [44.83333974000826]
  We introduce a minimalist benchmark that isolates long-range retrieval through pure second-order correlation signals.<n> Numerical investigations show that this task retains key qualitative aspects of how Transformers develop match-and-copy circuits.<n>We prove this max-margin alignment for GD trajectories that reach vanishing empirical loss under explicit technical conditions.
  arXiv  Detail & Related papers  (2026-02-07T14:18:11Z)
• Deep Generative Symbolic Regression [83.04219479605801]
  Symbolic regression aims to discover concise closed-form mathematical equations from data. Existing methods, ranging from search to reinforcement learning, fail to scale with the number of input variables. We propose an instantiation of our framework, Deep Generative Symbolic Regression.
  arXiv  Detail & Related papers  (2023-12-30T17:05:31Z)
• Understanding Augmentation-based Self-Supervised Representation Learning via RKHS Approximation and Regression [53.15502562048627]
  Recent work has built the connection between self-supervised learning and the approximation of the top eigenspace of a graph Laplacian operator. This work delves into a statistical analysis of augmentation-based pretraining.
  arXiv  Detail & Related papers  (2023-06-01T15:18:55Z)
• Scalable Learning of Latent Language Structure With Logical Offline Cycle Consistency [71.42261918225773]
  Conceptually, LOCCO can be viewed as a form of self-learning where the semantic being trained is used to generate annotations for unlabeled text. As an added bonus, the annotations produced by LOCCO can be trivially repurposed to train a neural text generation model.
  arXiv  Detail & Related papers  (2023-05-31T16:47:20Z)
• Variational Laplace Autoencoders [53.08170674326728]
  Variational autoencoders employ an amortized inference model to approximate the posterior of latent variables. We present a novel approach that addresses the limited posterior expressiveness of fully-factorized Gaussian assumption. We also present a general framework named Variational Laplace Autoencoders (VLAEs) for training deep generative models.
  arXiv  Detail & Related papers  (2022-11-30T18:59:27Z)
• Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognition [62.83832841523525]
  We propose a fast and accurate parallel transformer, termed Paraformer. It accurately predicts the number of output tokens and extract hidden variables. It can attain comparable performance to the state-of-the-art AR transformer, with more than 10x speedup.
  arXiv  Detail & Related papers  (2022-06-16T17:24:14Z)
• Representation Learning via Cauchy Convolutional Sparse Coding [7.784354802855044]
  Convolutional Sparse Coding (CSC) enables unsupervised learning of features by jointly optimising both an (ell)-norm fidelity term and a sparsity enforcing penalty. This work investigates using a regularisation term derived from an assumed Cauchy prior for the coefficients of the feature maps of a CSC generative model. The sparsity penalty term resulting from this prior is solved via its proximal operator, which is then applied iteratively, element-wise, on the coefficients of the feature maps to optimise the CSC cost function.
  arXiv  Detail & Related papers  (2020-08-08T08:21:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.