Transformer Semantic Genetic Programming for Symbolic Regression

• URL: http://arxiv.org/abs/2501.18479v1
• Date: Thu, 30 Jan 2025 16:51:44 GMT
• Title: Transformer Semantic Genetic Programming for Symbolic Regression
• Authors: Philipp Anthes, Dominik Sobania, Franz Rothlauf,
• Abstract summary: Transformer Genetic Programming (TSGP) is a novel and flexible semantic approach that uses a generative transformer model as search operator. The model is trained on synthetic test problems and learns semantic similarities between solutions. An analysis of the search dynamic reveals that the solutions generated by TSGP are semantically more similar than the solutions generated by the benchmark approaches.
• Score: 0.8602553195689513
• License:
• Abstract: In standard genetic programming (stdGP), solutions are varied by modifying their syntax, with uncertain effects on their semantics. Geometric-semantic genetic programming (GSGP), a popular variant of GP, effectively searches the semantic solution space using variation operations based on linear combinations, although it results in significantly larger solutions. This paper presents Transformer Semantic Genetic Programming (TSGP), a novel and flexible semantic approach that uses a generative transformer model as search operator. The transformer is trained on synthetic test problems and learns semantic similarities between solutions. Once the model is trained, it can be used to create offspring solutions with high semantic similarity also for unseen and unknown problems. Experiments on several symbolic regression problems show that TSGP generates solutions with comparable or even significantly better prediction quality than stdGP, SLIM_GSGP, DSR, and DAE-GP. Like SLIM_GSGP, TSGP is able to create new solutions that are semantically similar without creating solutions of large size. An analysis of the search dynamic reveals that the solutions generated by TSGP are semantically more similar than the solutions generated by the benchmark approaches allowing a better exploration of the semantic solution space.

Related papers

• Deep Transformed Gaussian Processes [0.0]
  Transformed Gaussian Processes (TGPs) are processes specified by transforming samples from the joint distribution from a prior process (typically a GP) using an invertible transformation. We propose a generalization of TGPs named Deep Transformed Gaussian Processes (DTGPs), which follows the trend of concatenating layers of processes. Experiments conducted evaluate the proposed DTGPs in multiple regression datasets, achieving good scalability and performance.
  arXiv  Detail & Related papers  (2023-10-27T16:09:39Z)
• Local Search, Semantics, and Genetic Programming: a Global Analysis [7.486818142115522]
  Geometric Semantic Geometric Programming (GSGP) is one of the most prominent Genetic Programming (GP) variants. Here we explore multiple possibilities to limit the overfitting of GSM-LS and GSGP-reg. Results show that it is possible to consistently outperform standard GSGP on both training and unseen data.
  arXiv  Detail & Related papers  (2023-05-26T14:13:03Z)
• Stochastic Gradient Descent-Ascent: Unified Theory and New Efficient Methods [73.35353358543507]
  Gradient Descent-Ascent (SGDA) is one of the most prominent algorithms for solving min-max optimization and variational inequalities problems (VIP) In this paper, we propose a unified convergence analysis that covers a large variety of descent-ascent methods. We develop several new variants of SGDA such as a new variance-reduced method (L-SVRGDA), new distributed methods with compression (QSGDA, DIANA-SGDA, VR-DIANA-SGDA), and a new method with coordinate randomization (SEGA-SGDA)
  arXiv  Detail & Related papers  (2022-02-15T09:17:39Z)
• Result Diversification by Multi-objective Evolutionary Algorithms with Theoretical Guarantees [94.72461292387146]
  We propose to reformulate the result diversification problem as a bi-objective search problem, and solve it by a multi-objective evolutionary algorithm (EA) We theoretically prove that the GSEMO can achieve the optimal-time approximation ratio, $1/2$. When the objective function changes dynamically, the GSEMO can maintain this approximation ratio in running time, addressing the open question proposed by Borodin et al.
  arXiv  Detail & Related papers  (2021-10-18T14:00:22Z)
• Incremental Ensemble Gaussian Processes [53.3291389385672]
  We propose an incremental ensemble (IE-) GP framework, where an EGP meta-learner employs an it ensemble of GP learners, each having a unique kernel belonging to a prescribed kernel dictionary. With each GP expert leveraging the random feature-based approximation to perform online prediction and model update with it scalability, the EGP meta-learner capitalizes on data-adaptive weights to synthesize the per-expert predictions. The novel IE-GP is generalized to accommodate time-varying functions by modeling structured dynamics at the EGP meta-learner and within each GP learner.
  arXiv  Detail & Related papers  (2021-10-13T15:11:25Z)
• Using Traceless Genetic Programming for Solving Multiobjective Optimization Problems [1.9493449206135294]
  Traceless Genetic Programming (TGP) is a Genetic Programming (GP) variant that is used in cases where the focus is rather the output of the program than the program itself. Two genetic operators are used in conjunction with TGP: crossover and insertion. Numerical experiments show that TGP is able to solve very fast and very well the considered test problems.
  arXiv  Detail & Related papers  (2021-10-07T05:55:55Z)
• Fractal Structure and Generalization Properties of Stochastic Optimization Algorithms [71.62575565990502]
  We prove that the generalization error of an optimization algorithm can be bounded on the complexity' of the fractal structure that underlies its generalization measure. We further specialize our results to specific problems (e.g., linear/logistic regression, one hidden/layered neural networks) and algorithms.
  arXiv  Detail & Related papers  (2021-06-09T08:05:36Z)
• Zoetrope Genetic Programming for Regression [2.642406403099596]
  The Zoetrope Genetic Programming (ZGP) algorithm is based on an original representation for mathematical expressions. ZGP is validated using a large number of public domain regression datasets.
  arXiv  Detail & Related papers  (2021-02-26T10:47:10Z)
• The data-driven physical-based equations discovery using evolutionary approach [77.34726150561087]
  We describe the algorithm for the mathematical equations discovery from the given observations data. The algorithm combines genetic programming with the sparse regression. It could be used for governing analytical equation discovery as well as for partial differential equations (PDE) discovery.
  arXiv  Detail & Related papers  (2020-04-03T17:21:57Z)
• SGP-DT: Semantic Genetic Programming Based on Dynamic Targets [6.841231589814175]
  This paper presents a new Semantic GP approach based on Dynamic Target (SGP-DT) The evolution in each run is guided by a new (dynamic) target based on the residual errors. SGP-DT achieves small RMSE values, on average 23.19% smaller than the one of epsilon-lexicase.
  arXiv  Detail & Related papers  (2020-01-30T19:33:58Z)
• Gated Path Selection Network for Semantic Segmentation [72.44994579325822]
  We develop a novel network named Gated Path Selection Network (GPSNet), which aims to learn adaptive receptive fields. In GPSNet, we first design a two-dimensional multi-scale network - SuperNet, which densely incorporates features from growing receptive fields. To dynamically select desirable semantic context, a gate prediction module is further introduced.
  arXiv  Detail & Related papers  (2020-01-19T12:32:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.