Deep Generative Symbolic Regression with Monte-Carlo-Tree-Search

• URL: http://arxiv.org/abs/2302.11223v2
• Date: Wed, 10 May 2023 16:20:24 GMT
• Title: Deep Generative Symbolic Regression with Monte-Carlo-Tree-Search
• Authors: Pierre-Alexandre Kamienny, Guillaume Lample, Sylvain Lamprier, Marco Virgolin
• Abstract summary: Symbolic regression is a problem of learning a symbolic expression from numerical data. Deep neural models trained on procedurally-generated synthetic datasets showed competitive performance. We propose a novel method which provides the best of both worlds, based on a Monte-Carlo Tree Search procedure.
• Score: 29.392036559507755
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Symbolic regression (SR) is the problem of learning a symbolic expression from numerical data. Recently, deep neural models trained on procedurally-generated synthetic datasets showed competitive performance compared to more classical Genetic Programming (GP) algorithms. Unlike their GP counterparts, these neural approaches are trained to generate expressions from datasets given as context. This allows them to produce accurate expressions in a single forward pass at test time. However, they usually do not benefit from search abilities, which result in low performance compared to GP on out-of-distribution datasets. In this paper, we propose a novel method which provides the best of both worlds, based on a Monte-Carlo Tree Search procedure using a context-aware neural mutation model, which is initially pre-trained to learn promising mutations, and further refined from successful experiences in an online fashion. The approach demonstrates state-of-the-art performance on the well-known \texttt{SRBench} benchmark.

Related papers

• A Comparison of Recent Algorithms for Symbolic Regression to Genetic Programming [0.0]
  Symbolic regression aims to model and map data in a way that can be understood by scientists. Recent advancements, have attempted to bridge the gap between these two fields.
  arXiv  Detail & Related papers  (2024-06-05T19:01:43Z)
• Generative Pre-Trained Transformer for Symbolic Regression Base In-Context Reinforcement Learning [12.660401635672967]
  Finding mathematical formulas from observational data is a major demand of scientific research. FormulaGPT achieves the state-of-the-art performance in fitting ability compared with four baselines.
  arXiv  Detail & Related papers  (2024-04-09T14:08:47Z)
• Noisy Self-Training with Synthetic Queries for Dense Retrieval [49.49928764695172]
  We introduce a novel noisy self-training framework combined with synthetic queries. Experimental results show that our method improves consistently over existing methods. Our method is data efficient and outperforms competitive baselines.
  arXiv  Detail & Related papers  (2023-11-27T06:19:50Z)
• What learning algorithm is in-context learning? Investigations with linear models [87.91612418166464]
  We investigate the hypothesis that transformer-based in-context learners implement standard learning algorithms implicitly. We show that trained in-context learners closely match the predictors computed by gradient descent, ridge regression, and exact least-squares regression. Preliminary evidence that in-context learners share algorithmic features with these predictors.
  arXiv  Detail & Related papers  (2022-11-28T18:59:51Z)
• Fast variable selection makes scalable Gaussian process BSS-ANOVA a speedy and accurate choice for tabular and time series regression [0.0]
  Gaussian processes (GPs) are non-parametric regression engines with a long history. One of a number of scalable GP approaches is the Karhunen-Lo'eve (KL) decomposed kernel BSS-ANOVA, developed in 2009. A new method of forward variable selection, quickly and effectively limits the number of terms, yielding a method with competitive accuracies.
  arXiv  Detail & Related papers  (2022-05-26T23:41:43Z)
• An Empirical Analysis of Recurrent Learning Algorithms In Neural Lossy Image Compression Systems [73.48927855855219]
  Recent advances in deep learning have resulted in image compression algorithms that outperform JPEG and JPEG 2000 on the standard Kodak benchmark. In this paper, we perform the first large-scale comparison of recent state-of-the-art hybrid neural compression algorithms.
  arXiv  Detail & Related papers  (2022-01-27T19:47:51Z)
• Symbolic Regression via Neural-Guided Genetic Programming Population Seeding [6.9501458586819505]
  Symbolic regression is a discrete optimization problem generally believed to be NP-hard. Prior approaches to solving the problem include neural-guided search and genetic programming. We propose a neural-guided component used to seed the starting population of a random restart genetic programming component.
  arXiv  Detail & Related papers  (2021-10-29T19:26:41Z)
• Gone Fishing: Neural Active Learning with Fisher Embeddings [55.08537975896764]
  There is an increasing need for active learning algorithms that are compatible with deep neural networks. This article introduces BAIT, a practical representation of tractable, and high-performing active learning algorithm for neural networks.
  arXiv  Detail & Related papers  (2021-06-17T17:26:31Z)
• Zoetrope Genetic Programming for Regression [2.642406403099596]
  The Zoetrope Genetic Programming (ZGP) algorithm is based on an original representation for mathematical expressions. ZGP is validated using a large number of public domain regression datasets.
  arXiv  Detail & Related papers  (2021-02-26T10:47:10Z)
• Learning Reasoning Strategies in End-to-End Differentiable Proving [50.9791149533921]
  Conditional Theorem Provers learn optimal rule selection strategy via gradient-based optimisation. We show that Conditional Theorem Provers are scalable and yield state-of-the-art results on the CLUTRR dataset.
  arXiv  Detail & Related papers  (2020-07-13T16:22:14Z)
• Closed Loop Neural-Symbolic Learning via Integrating Neural Perception, Grammar Parsing, and Symbolic Reasoning [134.77207192945053]
  Prior methods learn the neural-symbolic models using reinforcement learning approaches. We introduce the textbfgrammar model as a textitsymbolic prior to bridge neural perception and symbolic reasoning. We propose a novel textbfback-search algorithm which mimics the top-down human-like learning procedure to propagate the error.
  arXiv  Detail & Related papers  (2020-06-11T17:42:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.