Symbolic Regression with Multimodal Large Language Models and Kolmogorov Arnold Networks

• URL: http://arxiv.org/abs/2505.07956v1
• Date: Mon, 12 May 2025 18:00:41 GMT
• Title: Symbolic Regression with Multimodal Large Language Models and Kolmogorov Arnold Networks
• Authors: Thomas R. Harvey, Fabian Ruehle, Cristofero S. Fraser-Taliente, James Halverson,
• Abstract summary: We present a novel approach to symbolic regression using vision-capable large language models (LLMs)<n>The method does not require the specification of a set of functions to be used in regression, but with appropriate prompt engineering, we can arbitrarily condition the generative step.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present a novel approach to symbolic regression using vision-capable large language models (LLMs) and the ideas behind Google DeepMind's Funsearch. The LLM is given a plot of a univariate function and tasked with proposing an ansatz for that function. The free parameters of the ansatz are fitted using standard numerical optimisers, and a collection of such ans\"atze make up the population of a genetic algorithm. Unlike other symbolic regression techniques, our method does not require the specification of a set of functions to be used in regression, but with appropriate prompt engineering, we can arbitrarily condition the generative step. By using Kolmogorov Arnold Networks (KANs), we demonstrate that ``univariate is all you need'' for symbolic regression, and extend this method to multivariate functions by learning the univariate function on each edge of a trained KAN. The combined expression is then simplified by further processing with a language model.

Related papers

• $p$-Adic Polynomial Regression as Alternative to Neural Network for Approximating $p$-Adic Functions of Many Variables [55.2480439325792]
  A regression model is constructed that allows approximating continuous functions with any degree of accuracy.<n>The proposed model can be considered as a simple alternative to possible $p$-adic models based on neural network architecture.
  arXiv  Detail & Related papers  (2025-03-30T15:42:08Z)
• Universal Neural Functionals [67.80283995795985]
  A challenging problem in many modern machine learning tasks is to process weight-space features. Recent works have developed promising weight-space models that are equivariant to the permutation symmetries of simple feedforward networks. This work proposes an algorithm that automatically constructs permutation equivariant models for any weight space.
  arXiv  Detail & Related papers  (2024-02-07T20:12:27Z)
• Deep Generative Symbolic Regression [83.04219479605801]
  Symbolic regression aims to discover concise closed-form mathematical equations from data. Existing methods, ranging from search to reinforcement learning, fail to scale with the number of input variables. We propose an instantiation of our framework, Deep Generative Symbolic Regression.
  arXiv  Detail & Related papers  (2023-12-30T17:05:31Z)
• GINN-LP: A Growing Interpretable Neural Network for Discovering Multivariate Laurent Polynomial Equations [1.1142444517901018]
  We propose GINN-LP, an interpretable neural network, to discover the form of a Laurent Polynomial equation. To the best of our knowledge, this is the first neural network that can discover arbitrary terms without any prior information on the order. We show that GINN-LP outperforms the state-of-theart symbolic regression methods on datasets.
  arXiv  Detail & Related papers  (2023-12-18T03:44:29Z)
• Meaning Representations from Trajectories in Autoregressive Models [106.63181745054571]
  We propose to extract meaning representations from autoregressive language models by considering the distribution of all possible trajectories extending an input text. This strategy is prompt-free, does not require fine-tuning, and is applicable to any pre-trained autoregressive model. We empirically show that the representations obtained from large models align well with human annotations, outperform other zero-shot and prompt-free methods on semantic similarity tasks, and can be used to solve more complex entailment and containment tasks that standard embeddings cannot handle.
  arXiv  Detail & Related papers  (2023-10-23T04:35:58Z)
• Amortizing intractable inference in large language models [56.92471123778389]
  We use amortized Bayesian inference to sample from intractable posterior distributions. We empirically demonstrate that this distribution-matching paradigm of LLM fine-tuning can serve as an effective alternative to maximum-likelihood training. As an important application, we interpret chain-of-thought reasoning as a latent variable modeling problem.
  arXiv  Detail & Related papers  (2023-10-06T16:36:08Z)
• Scalable Neural Symbolic Regression using Control Variables [7.725394912527969]
  We propose ScaleSR, a scalable symbolic regression model that leverages control variables to enhance both accuracy and scalability. The proposed method involves a four-step process. First, we learn a data generator from observed data using deep neural networks (DNNs) Experimental results demonstrate that the proposed ScaleSR significantly outperforms state-of-the-art baselines in discovering mathematical expressions with multiple variables.
  arXiv  Detail & Related papers  (2023-06-07T18:30:25Z)
• An Iterative Algorithm for Rescaled Hyperbolic Functions Regression [7.578147116161996]
  Large language models (LLMs) have numerous real-life applications across various domains.<n>LLMs have the potential to revolutionize the field of natural language processing (NLP)
  arXiv  Detail & Related papers  (2023-05-01T05:16:07Z)
• Symbolic Metamodels for Interpreting Black-boxes Using Primitive Functions [15.727276506140878]
  One approach for interpreting black-box machine learning models is to find a global approximation of the model using simple interpretable functions. In this work, we propose a new method for finding interpretable metamodels.
  arXiv  Detail & Related papers  (2023-02-09T17:30:43Z)
• Learning Symbolic Expressions: Mixed-Integer Formulations, Cuts, and Heuristics [1.1602089225841632]
  We consider the problem of learning a regression function without assuming its functional form. We propose a that builds an expression tree by solving a restricted MI.
  arXiv  Detail & Related papers  (2021-02-16T18:39:14Z)
• The data-driven physical-based equations discovery using evolutionary approach [77.34726150561087]
  We describe the algorithm for the mathematical equations discovery from the given observations data. The algorithm combines genetic programming with the sparse regression. It could be used for governing analytical equation discovery as well as for partial differential equations (PDE) discovery.
  arXiv  Detail & Related papers  (2020-04-03T17:21:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.