Decoupling Representation and Learning in Genetic Programming: the LaSER Approach

• URL: http://arxiv.org/abs/2505.17309v2
• Date: Fri, 06 Jun 2025 08:30:09 GMT
• Title: Decoupling Representation and Learning in Genetic Programming: the LaSER Approach
• Abstract summary: We propose LaSER -- a framework that decouples representation evolution from lifetime learning.<n>We show for the first time that LaSER can outcompete standard GP and GP followed by linear regression.<n>By separating the roles of representation and adaptation, LaSER offers a principled and framework for symbolic regression and classification.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Genetic Programming (GP) has traditionally entangled the evolution of symbolic representations with their performance-based evaluation, often relying solely on raw fitness scores. This tight coupling makes GP solutions more fragile and prone to overfitting, reducing their ability to generalize. In this work, we propose LaSER (Latent Semantic Representation Regression)} -- a general framework that decouples representation evolution from lifetime learning. At each generation, candidate programs produce features which are passed to an external learner to model the target task. This approach enables any function approximator, from linear models to neural networks, to serve as a lifetime learner, allowing expressive modeling beyond conventional symbolic forms. Here we show for the first time that LaSER can outcompete standard GP and GP followed by linear regression when it employs non-linear methods to fit coefficients to GP-generated equations against complex data sets. Further, we explore how LaSER enables the emergence of innate representations, supporting long-standing hypotheses in evolutionary learning such as the Baldwin Effect. By separating the roles of representation and adaptation, LaSER offers a principled and extensible framework for symbolic regression and classification.

Related papers

• Influence-Guided Symbolic Regression: Scientific Discovery via LLM-Driven Equation Search with Granular Feedback [56.69850045068714]
  We introduce textitInfluence-Guided Regression (IGSR), a method that frames equation discovery as an iterative two-step process.<n>We demonstrate IGSRs effectiveness on a diverse suite of benchmarks, including LLM-SRBench, pharmacological PKPD models, an epidemiological simulation, and real-world genomic data.
  arXiv  Detail & Related papers  (2026-05-27T23:48:01Z)
• Power Reinforcement Post-Training of Text-to-Image Models with Super-Linear Advantage Shaping [66.25536973294726]
  Post-training methods for text-to-image (T2I) models are prone to reward hacking.<n>Super-Linear Advantage Shaping (SLAS) reshapes the local policy space.<n>SLAS consistently surpasses the DanceGRPO baseline across multiple backbones and benchmarks.
  arXiv  Detail & Related papers  (2026-05-11T17:59:25Z)
• Generalization in Representation Models via Random Matrix Theory: Application to Recurrent Networks [7.721672385781673]
  We first study the generalization error of models that use a fixed feature representation (frozen intermediate layers) followed by a trainable readout layer.<n>We apply Random Matrix Theory to derive a closed-form expression for the generalization error.<n>We then apply this analysis to recurrent representations and obtain concise formula that characterize their performance.
  arXiv  Detail & Related papers  (2025-11-04T09:30:31Z)
• Deep Equilibrium models for Poisson Imaging Inverse problems via Mirror Descent [7.248102801711294]
  Deep Equilibrium Models (DEQs) are implicit neural networks with fixed points.<n>We introduce a novel DEQ formulation based on Mirror Descent defined in terms of a tailored non-Euclidean geometry.<n>We propose computational strategies that enable both efficient training and fully parameter-free inference.
  arXiv  Detail & Related papers  (2025-07-15T16:33:01Z)
• Self-Boost via Optimal Retraining: An Analysis via Approximate Message Passing [58.52119063742121]
  Retraining a model using its own predictions together with the original, potentially noisy labels is a well-known strategy for improving the model performance.<n>This paper addresses the question of how to optimally combine the model's predictions and the provided labels.<n>Our main contribution is the derivation of the Bayes optimal aggregator function to combine the current model's predictions and the given labels.
  arXiv  Detail & Related papers  (2025-05-21T07:16:44Z)
• Koopman-Equivariant Gaussian Processes [39.34668284375732]
  We propose a family of Gaussian processes (GP) for dynamical systems with linear time-invariant responses.<n>This linearity allows us to tractably quantify forecasting and representational uncertainty.<n>Experiments demonstrate on-par and often better forecasting performance compared to kernel-based methods for learning dynamical systems.
  arXiv  Detail & Related papers  (2025-02-10T16:35:08Z)
• Universal In-Context Approximation By Prompting Fully Recurrent Models [86.61942787684272]
  We show that RNNs, LSTMs, GRUs, Linear RNNs, and linear gated architectures can serve as universal in-context approximators. We introduce a programming language called LSRL that compiles to fully recurrent architectures.
  arXiv  Detail & Related papers  (2024-06-03T15:25:13Z)
• VQDNA: Unleashing the Power of Vector Quantization for Multi-Species Genomic Sequence Modeling [60.91599380893732]
  VQDNA is a general-purpose framework that renovates genome tokenization from the perspective of genome vocabulary learning. By leveraging vector-quantized codebooks as learnable vocabulary, VQDNA can adaptively tokenize genomes into pattern-aware embeddings.
  arXiv  Detail & Related papers  (2024-05-13T20:15:03Z)
• Scaling and renormalization in high-dimensional regression [72.59731158970894]
  We present a unifying perspective on recent results on ridge regression.<n>We use the basic tools of random matrix theory and free probability, aimed at readers with backgrounds in physics and deep learning.<n>Our results extend and provide a unifying perspective on earlier models of scaling laws.
  arXiv  Detail & Related papers  (2024-05-01T15:59:00Z)
• Evolving Reservoirs for Meta Reinforcement Learning [1.6874375111244329]
  We propose a computational model for studying a mechanism that can enable such a process. At the evolutionary scale, we evolve reservoirs, a family of recurrent neural networks. We employ these evolved reservoirs to facilitate the learning of a behavioral policy through Reinforcement Learning (RL) Our results show that the evolution of reservoirs can improve the learning of diverse challenging tasks.
  arXiv  Detail & Related papers  (2023-12-09T16:11:48Z)
• Linear Time GPs for Inferring Latent Trajectories from Neural Spike Trains [7.936841911281107]
  We propose cvHM, a general inference framework for latent GP models leveraging Hida-Mat'ern kernels and conjugate variational inference (CVI) We are able to perform variational inference of latent neural trajectories with linear time complexity for arbitrary likelihoods.
  arXiv  Detail & Related papers  (2023-06-01T16:31:36Z)
• Theoretical Characterization of the Generalization Performance of Overfitted Meta-Learning [70.52689048213398]
  This paper studies the performance of overfitted meta-learning under a linear regression model with Gaussian features. We find new and interesting properties that do not exist in single-task linear regression. Our analysis suggests that benign overfitting is more significant and easier to observe when the noise and the diversity/fluctuation of the ground truth of each training task are large.
  arXiv  Detail & Related papers  (2023-04-09T20:36:13Z)
• Deep Generative Symbolic Regression with Monte-Carlo-Tree-Search [29.392036559507755]
  Symbolic regression is a problem of learning a symbolic expression from numerical data. Deep neural models trained on procedurally-generated synthetic datasets showed competitive performance. We propose a novel method which provides the best of both worlds, based on a Monte-Carlo Tree Search procedure.
  arXiv  Detail & Related papers  (2023-02-22T09:10:20Z)
• Equivariance with Learned Canonicalization Functions [77.32483958400282]
  We show that learning a small neural network to perform canonicalization is better than using predefineds. Our experiments show that learning the canonicalization function is competitive with existing techniques for learning equivariant functions across many tasks.
  arXiv  Detail & Related papers  (2022-11-11T21:58:15Z)
• Complex Evolutional Pattern Learning for Temporal Knowledge Graph Reasoning [60.94357727688448]
  TKG reasoning aims to predict potential facts in the future given the historical KG sequences. The evolutional patterns are complex in two aspects, length-diversity and time-variability. We propose a new model, called Complex Evolutional Network (CEN), which uses a length-aware Convolutional Neural Network (CNN) to handle evolutional patterns of different lengths.
  arXiv  Detail & Related papers  (2022-03-15T11:02:55Z)
• Transfer Dynamics in Emergent Evolutionary Curricula [4.692078300163222]
  PINSKY is a system for open-ended learning through neuroevolution in game-based domains. This paper focuses on the role of transfer of policies from one evolutionary branch ("species") to another. The most insightful finding is that inter-species transfer, while rare, is crucial to the system's success.
  arXiv  Detail & Related papers  (2022-03-03T21:10:22Z)
• Heritability in Morphological Robot Evolution [2.7402733069181]
  We introduce the biological notion of heritability, which captures the amount of phenotypic variation caused by genotypic variation. In our analysis we measure the heritability on the first generation of robots evolved from two different encodings. We show how heritability can be a useful tool to better understand the relationship between genotypes and phenotypes.
  arXiv  Detail & Related papers  (2021-10-21T14:58:17Z)
• Incremental Ensemble Gaussian Processes [53.3291389385672]
  We propose an incremental ensemble (IE-) GP framework, where an EGP meta-learner employs an it ensemble of GP learners, each having a unique kernel belonging to a prescribed kernel dictionary. With each GP expert leveraging the random feature-based approximation to perform online prediction and model update with it scalability, the EGP meta-learner capitalizes on data-adaptive weights to synthesize the per-expert predictions. The novel IE-GP is generalized to accommodate time-varying functions by modeling structured dynamics at the EGP meta-learner and within each GP learner.
  arXiv  Detail & Related papers  (2021-10-13T15:11:25Z)
• Epigenetic opportunities for Evolutionary Computation [0.0]
  Evolutionary Computation is a group of biologically inspired algorithms used to solve complex optimisation problems. It can be split into Evolutionary Algorithms, which take inspiration from genetic inheritance, and Swarm Intelligence algorithms, that take inspiration from cultural inheritance. This paper breaks down successful bio-inspired algorithms under a contemporary biological framework based on the Extended Evolutionary Synthesis.
  arXiv  Detail & Related papers  (2021-08-10T09:44:53Z)
• Learning an Explicit Hyperparameter Prediction Function Conditioned on Tasks [62.63852372239708]
  Meta learning aims to learn the learning methodology for machine learning from observed tasks, so as to generalize to new query tasks. We interpret such learning methodology as learning an explicit hyper- parameter prediction function shared by all training tasks. Such setting guarantees that the meta-learned learning methodology is able to flexibly fit diverse query tasks.
  arXiv  Detail & Related papers  (2021-07-06T04:05:08Z)
• Epigenetic evolution of deep convolutional models [81.21462458089142]
  We build upon a previously proposed neuroevolution framework to evolve deep convolutional models. We propose a convolutional layer layout which allows kernels of different shapes and sizes to coexist within the same layer. The proposed layout enables the size and shape of individual kernels within a convolutional layer to be evolved with a corresponding new mutation operator.
  arXiv  Detail & Related papers  (2021-04-12T12:45:16Z)
• Generalization Error of Generalized Linear Models in High Dimensions [25.635225717360466]
  We provide a framework to characterize neural networks with arbitrary non-linearities. We analyze the effect of regular logistic regression on learning. Our model also captures examples between training and distributions special cases.
  arXiv  Detail & Related papers  (2020-05-01T02:17:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.