Define latent spaces by example: optimisation over the outputs of generative models

• URL: http://arxiv.org/abs/2509.23800v1
• Date: Sun, 28 Sep 2025 10:50:06 GMT
• Title: Define latent spaces by example: optimisation over the outputs of generative models
• Authors: Samuel Willis, Alexandru I. Stere, Dragos D. Margineantu, Henry T. Oldroyd, John A. Fozard, Carl Henrik Ek, Henry Moss, Erik Bodin,
• Abstract summary: Many downstream tasks require a higher level of control than unconstrained sampling.<n>We introduce surrogate latent spaces: non-parametric, low-dimensional Euclidean embeddings that can be extracted from any generative model.<n>Our approach is architecture-agnostic, incurs almost no additional computational cost, and generalises across modalities.
• Score: 37.62017041960412
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modern generative AI models such as diffusion and flow matching can sample from rich data distributions, but many downstream tasks -- such as experimental design or creative content generation -- require a higher level of control than unconstrained sampling. The challenge is to efficiently identify outputs that are both probable under the model and satisfy task-specific constraints. We address this by introducing surrogate latent spaces: non-parametric, low-dimensional Euclidean embeddings that can be extracted from any generative model without additional training. The axes in the Euclidean space can be defined via examples, providing a simple and interpretable approach to define custom latent spaces that both express intended features and are convenient to use in downstream tasks. The representation is Euclidean and has controllable dimensionality, permitting direct application of standard optimisation algorithms to traverse the outputs of generative models. Our approach is architecture-agnostic, incurs almost no additional computational cost, and generalises across modalities, including images, audio, videos, and structured objects like proteins.

Related papers

• Composition and Alignment of Diffusion Models using Constrained Learning [79.36736636241564]
  Diffusion models have become prevalent in generative modeling due to their ability to sample from complex distributions.<n>Two commonly used methods are: (i) alignment, which involves fine-tuning a diffusion model to align it with a reward; and (ii) composition, which combines several pre-trained diffusion models, each emphasizing a desirable attribute in the generated outputs.<n>We propose a constrained optimization framework that unifies alignment and composition of diffusion models by enforcing that the aligned model satisfies reward constraints and/or remains close to (potentially multiple) pre-trained models.
  arXiv  Detail & Related papers  (2025-08-26T15:06:30Z)
• Geometric Regularity in Deterministic Sampling of Diffusion-based Generative Models [39.94246633953425]
  We reveal a striking geometric regularity in the deterministic sampling dynamics.<n>All trajectories exhibit an almost identical ''boomerang'' shape, regardless of the model architecture, applied conditions, or generated content.<n>We propose a dynamic programming-based scheme to better align the sampling time schedule with the underlying trajectory structure.
  arXiv  Detail & Related papers  (2025-06-11T21:09:09Z)
• Plug-and-Play Controllable Generation for Discrete Masked Models [27.416952690340903]
  This article makes discrete masked models for the generative modeling of discrete data controllable. We propose a novel plug-and-play framework based on importance sampling that bypasses the need for training a conditional score. Our framework is agnostic to the choice of control criteria, requires no gradient information, and is well-suited for tasks such as posterior sampling, Bayesian inverse problems, and constrained generation.
  arXiv  Detail & Related papers  (2024-10-03T02:00:40Z)
• Linear combinations of latents in generative models: subspaces and beyond [44.72663191923503]
  We propose Latent Optimal Linear combinations (LOL) as a general-purpose method to form linear combinations of latent variables.<n>LOL is easy to implement and naturally addresses the broader task of forming any linear combinations.<n>It dramatically simplifies the creation of expressive low-dimensional representations of high-dimensional objects.
  arXiv  Detail & Related papers  (2024-08-16T06:43:58Z)
• Diffusion Model for Data-Driven Black-Box Optimization [54.25693582870226]
  We focus on diffusion models, a powerful generative AI technology, and investigate their potential for black-box optimization. We study two practical types of labels: 1) noisy measurements of a real-valued reward function and 2) human preference based on pairwise comparisons. Our proposed method reformulates the design optimization problem into a conditional sampling problem, which allows us to leverage the power of diffusion models.
  arXiv  Detail & Related papers  (2024-03-20T00:41:12Z)
• Constrained Synthesis with Projected Diffusion Models [47.56192362295252]
  This paper introduces an approach to generative diffusion processes the ability to satisfy and certify compliance with constraints and physical principles. The proposed method recast the traditional process of generative diffusion as a constrained distribution problem to ensure adherence to constraints.
  arXiv  Detail & Related papers  (2024-02-05T22:18:16Z)
• Protein Design with Guided Discrete Diffusion [67.06148688398677]
  A popular approach to protein design is to combine a generative model with a discriminative model for conditional sampling. We propose diffusioN Optimized Sampling (NOS), a guidance method for discrete diffusion models. NOS makes it possible to perform design directly in sequence space, circumventing significant limitations of structure-based methods.
  arXiv  Detail & Related papers  (2023-05-31T16:31:24Z)
• Approximating Constraint Manifolds Using Generative Models for Sampling-Based Constrained Motion Planning [8.924344714683814]
  This paper presents a learning-based sampling strategy for constrained motion planning problems. We use Conditional Variversaational Autoencoder (CVAE) and Conditional Generative Adrial Net (CGAN) to generate constraint-satisfying sample configurations. We evaluate the efficiency of these two generative models in terms of their sampling accuracy and coverage of sampling distribution.
  arXiv  Detail & Related papers  (2022-04-14T07:08:30Z)
• Conditional Generative Modeling via Learning the Latent Space [54.620761775441046]
  We propose a novel framework for conditional generation in multimodal spaces. It uses latent variables to model generalizable learning patterns. At inference, the latent variables are optimized to find optimal solutions corresponding to multiple output modes.
  arXiv  Detail & Related papers  (2020-10-07T03:11:34Z)
• Goal-directed Generation of Discrete Structures with Conditional Generative Models [85.51463588099556]
  We introduce a novel approach to directly optimize a reinforcement learning objective, maximizing an expected reward. We test our methodology on two tasks: generating molecules with user-defined properties and identifying short python expressions which evaluate to a given target value.
  arXiv  Detail & Related papers  (2020-10-05T20:03:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.