Multi-Modal Learning meets Genetic Programming: Analyzing Alignment in Latent Space Optimization

• URL: http://arxiv.org/abs/2604.08324v2
• Date: Mon, 13 Apr 2026 17:02:13 GMT
• Title: Multi-Modal Learning meets Genetic Programming: Analyzing Alignment in Latent Space Optimization
• Abstract summary: Symbolic regression aims to discover mathematical expressions from data.<n>Latent Space Optimization methods use neural encoders to map symbolic expressions into continuous spaces.<n>SNIP advances LSO by introducing a multi-modal approach: aligning symbolic and numeric encoders in a shared latent space to learn the phenotype-genotype mapping.<n>This relies on fine-grained cross-modal alignment, whereas on similar models like CLIP reveals that such an alignment is typically coarse-grained.
• Score: 8.749115213097555
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Symbolic regression (SR) aims to discover mathematical expressions from data, a task traditionally tackled using Genetic Programming (GP) through combinatorial search over symbolic structures. Latent Space Optimization (LSO) methods use neural encoders to map symbolic expressions into continuous spaces, transforming the combinatorial search into continuous optimization. SNIP (Meidani et al., 2024), a contrastive pre-training model inspired by CLIP, advances LSO by introducing a multi-modal approach: aligning symbolic and numeric encoders in a shared latent space to learn the phenotype-genotype mapping, enabling optimization in the numeric space to implicitly guide symbolic search. However, this relies on fine-grained cross-modal alignment, whereas literature on similar models like CLIP reveals that such an alignment is typically coarse-grained. In this paper, we investigate whether SNIP delivers on its promise of effective bi-modal optimization for SR. Our experiments show that: (1) cross-modal alignment does not improve during optimization, even as fitness increases, and (2) the alignment learned by SNIP is too coarse to efficiently conduct principled search in the symbolic space. These findings reveal that while multi-modal LSO holds significant potential for SR, effective alignment-guided optimization remains unrealized in practice, highlighting fine-grained alignment as a critical direction for future work.

Related papers

• InsightSR: Refining Symbolic Regression Search Spaces via Parallel Semantic and Structural LLM Guidance [8.272456038469484]
  Symbolic regression seeks to discover parsimonious mathematical laws from observational data.<n>We present InsightSR, a framework that embeds Large Language Models (LLMs) iteration as a guiding layer around the PySR genetic programming engine.
  arXiv  Detail & Related papers  (2026-08-26T02:04:20Z)
• SMI: Efficient Self-Supervised Learning via Mutual-Information-Inspired Dependency Optimization [8.864968998286772]
  Self-supervised learning (SSL) has achieved remarkable representation learning performance.<n>Many existing methods rely on large batch sizes, memory banks, momentum encoders, or global synchronization mechanisms.<n>We propose Semantic Mutual Information (SMI), a lightweight self-supervised objective derived from a mutual-information-inspired dependency formulation.
  arXiv  Detail & Related papers  (2026-06-06T20:50:04Z)
• Q-BIOLAT: Binary Latent Protein Fitness Landscapes for QUBO-Based Optimization [8.081417415802443]
  We introduce Q-BIOLAT, a framework for modeling and optimizing protein fitness landscapes in compact binary latent spaces.<n>By expressing the objective in QUBO form, our approach connects modern machine learning with discrete and quantum-inspired optimization.
  arXiv  Detail & Related papers  (2026-03-29T05:33:55Z)
• RAPO++: Cross-Stage Prompt Optimization for Text-to-Video Generation via Data Alignment and Test-Time Scaling [59.088798018184235]
  textbfRAPO++ is a cross-stage prompt optimization framework.<n>It unifies training-data-aligned refinement, test-time iterative scaling, and large language model fine-tuning.<n> RAPO++ achieves significant gains in semantic alignment, compositional reasoning, temporal stability, and physical plausibility.
  arXiv  Detail & Related papers  (2025-10-23T04:45:09Z)
• Unlocking Symbol-Level Precoding Efficiency Through Tensor Equivariant Neural Network [84.22115118596741]
  We propose an end-to-end deep learning (DL) framework with low inference complexity for symbol-level precoding.<n>We show that the proposed framework captures substantial performance gains of optimal SLP, while achieving an approximately 80-times speedup over conventional methods.
  arXiv  Detail & Related papers  (2025-10-02T15:15:50Z)
• L-SR1: Learned Symmetric-Rank-One Preconditioning [5.421390145168128]
  End-to-end deep learning has achieved impressive results but remains limited by its reliance on large labeled datasets.<n>In contrast, classical optimization methods are data-efficient and lightweight but often suffer from slow convergence.<n>We propose a novel learned second-order vectors that introduces a trainable preconditioning unit to enhance the classical Symmetric-Rank-One algorithm.
  arXiv  Detail & Related papers  (2025-08-17T07:37:29Z)
• Deep Symbolic Optimization: Reinforcement Learning for Symbolic Mathematics [43.622135148720886]
  Deep Symbolic Optimization (DSO) is a novel computational framework that enables symbolic optimization for scientific discovery.<n>One notable example is equation discovery, which aims to automatically derive mathematical models expressed in symbolic form.<n>In this chapter, we provide a comprehensive overview of the DSO framework and illustrate its transformative potential for automating symbolic optimization in scientific discovery.
  arXiv  Detail & Related papers  (2025-05-16T00:31:19Z)
• Federated Conditional Stochastic Optimization [110.513884892319]
  Conditional optimization has found in a wide range of machine learning tasks, such as in-variant learning tasks, AUPRC, andAML. This paper proposes algorithms for distributed federated learning.
  arXiv  Detail & Related papers  (2023-10-04T01:47:37Z)
• Performance Embeddings: A Similarity-based Approach to Automatic Performance Optimization [71.69092462147292]
  Performance embeddings enable knowledge transfer of performance tuning between applications. We demonstrate this transfer tuning approach on case studies in deep neural networks, dense and sparse linear algebra compositions, and numerical weather prediction stencils.
  arXiv  Detail & Related papers  (2023-03-14T15:51:35Z)
• CR-LSO: Convex Neural Architecture Optimization in the Latent Space of Graph Variational Autoencoder with Input Convex Neural Networks [6.026956571669411]
  In neural architecture search (NAS) methods based on latent space optimization (LSO), a deep generative model is trained to embed discrete neural architectures into a continuous latent space.<n>This paper develops a convexity architecture regularized space (CRLSO) method, which aims to regularize the learning process of space in order to obtain a convex performance mapping.<n> Experimental results on three popular NAS benchmarks show that CR-LSO achieves competitive evaluation results in terms of both computational complexity and performance.
  arXiv  Detail & Related papers  (2022-11-11T01:55:11Z)
• Tree ensemble kernels for Bayesian optimization with known constraints over mixed-feature spaces [54.58348769621782]
  Tree ensembles can be well-suited for black-box optimization tasks such as algorithm tuning and neural architecture search. Two well-known challenges in using tree ensembles for black-box optimization are (i) effectively quantifying model uncertainty for exploration and (ii) optimizing over the piece-wise constant acquisition function. Our framework performs as well as state-of-the-art methods for unconstrained black-box optimization over continuous/discrete features and outperforms competing methods for problems combining mixed-variable feature spaces and known input constraints.
  arXiv  Detail & Related papers  (2022-07-02T16:59:37Z)
• Meta-Learning with Neural Tangent Kernels [58.06951624702086]
  We propose the first meta-learning paradigm in the Reproducing Kernel Hilbert Space (RKHS) induced by the meta-model's Neural Tangent Kernel (NTK) Within this paradigm, we introduce two meta-learning algorithms, which no longer need a sub-optimal iterative inner-loop adaptation as in the MAML framework. We achieve this goal by 1) replacing the adaptation with a fast-adaptive regularizer in the RKHS; and 2) solving the adaptation analytically based on the NTK theory.
  arXiv  Detail & Related papers  (2021-02-07T20:53:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.