Diffusing in the Right Space: A Systematic Study of Latent Diffusability

• URL: http://arxiv.org/abs/2606.03578v1
• Date: Tue, 02 Jun 2026 12:47:14 GMT
• Title: Diffusing in the Right Space: A Systematic Study of Latent Diffusability
• Abstract summary: Latent diffusion models leverage visual tokenizers to compress images into latent spaces for efficient generative modeling.<n>We conduct a systematic study of latent diffusability by training a large collection of tokenizers with diverse regularization strategies.<n>We identify several latent properties that consistently correlate with generation quality and exhibit strong generalization across experimental settings.
• Score: 11.450517054285788
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Latent diffusion models leverage visual tokenizers to compress images into latent spaces for efficient generative modeling. However, better reconstruction quality of a tokenizer does not necessarily translate into better generation quality, suggesting that latent representations should be evaluated not only by fidelity but also by their diffusability. Recent studies have proposed diverse explanations for diffusion-friendly latent spaces, including semantic separability, affine equivariance, distribution uniformity, spatial structure, spectral smoothness, and manifold continuity. Yet these properties are often validated on a limited set of tokenizers, leaving it unclear which factors are most predictive of downstream generation quality and whether such conclusions hold beyond the specific settings in which they are introduced. In this work, we conduct a systematic study of latent diffusability by training a large collection of tokenizers with diverse regularization strategies, architectures, and latent configurations, and evaluating them with multiple downstream diffusion backbones. Our analysis identifies several latent properties that consistently correlate with generation quality and exhibit strong generalization across experimental settings. Beyond existing metrics, we introduce Velocity Irreducible Variance (VIV), a measure of velocity ambiguity induced by trajectory crossings. Extensive experiments show that VIV is one of the most stable predictors of generation quality.

Related papers

• Disentangling Continuous-Time Latent Dynamics: Identifiability of Latent SDEs via Diffusion Shifts [60.95873409673775]
  identifiability in continuous-time latent differential equation models remains largely open.<n>We study additive-noise latent SDEs observed through an unknown nonlinear diffeomorphism.<n>We propose a two-stage estimator for latent disentanglement and optional graph recovery.
  arXiv  Detail & Related papers  (2026-06-26T16:18:28Z)
• Probability-Conserving Flow Guidance [49.03107678763765]
  Adaptive Manifold Guidance (AdaMaG) improves realism, reduces hallucinations, and controlled desaturation in high-guidance.<n>Across image generation benchmarks, AdaMaG improves realism, reduces hallucinations, and controlled desaturation in high-guidance.
  arXiv  Detail & Related papers  (2026-05-19T16:34:01Z)
• Provably Learning Diffusion Models under the Manifold Hypothesis: Collapse and Refine [60.669081685261965]
  Diffusion models generate high-dimensional data with remarkable quality.<n>How their training efficiently learns the score function remains theoretically unexplained.<n>We instantiate this principle as Score-induced Latent Diffusion (SiLD)
  arXiv  Detail & Related papers  (2026-05-16T16:51:10Z)
• Taming Sampling Perturbations with Variance Expansion Loss for Latent Diffusion Models [22.168463445392856]
  Latent diffusion models have emerged as the dominant framework for high-fidelity and efficient image generation.<n>We show that robustness to sampling perturbations plays a crucial role in determining generation quality.<n>We propose a solution that constructs a latent space robust to sampling perturbations while maintaining strong reconstruction.
  arXiv  Detail & Related papers  (2026-03-22T06:56:12Z)
• Unconditional flow-based time series generation with equivariance-regularised latent spaces [0.0]
  Flow-based models have proven successful for time-series generation.<n>However, how to design latent representations with desirable equivariance properties for time-series generative modelling remains underexplored.<n>We propose a latent flow-matching framework in which equivariance is explicitly encouraged through a simple regularisation of a pre-trained autoencoder.
  arXiv  Detail & Related papers  (2026-01-30T11:19:33Z)
• Divergence-Free Diffusion Models for Incompressible Fluid Flows [0.0]
  Generative diffusion models are extensively used in unsupervised and self-supervised machine learning.<n>This work investigates their application to the numerical simulation of incompressible fluid flows.
  arXiv  Detail & Related papers  (2026-01-27T08:49:26Z)
• CORAL: Disentangling Latent Representations in Long-Tailed Diffusion [4.310167974376405]
  We investigate the behavior of diffusion models trained on long-tailed datasets.<n>Latent representations for tail class subspaces exhibit significant overlap with those of head classes.<n>We propose a contrastive latent alignment framework that leverages supervised contrastive losses to encourage well-separated latent class representations.
  arXiv  Detail & Related papers  (2025-06-19T00:23:44Z)
• Solving Inverse Problems with FLAIR [68.87167940623318]
  We present FLAIR, a training-free variational framework that leverages flow-based generative models as prior for inverse problems.<n>Results on standard imaging benchmarks demonstrate that FLAIR consistently outperforms existing diffusion- and flow-based methods in terms of reconstruction quality and sample diversity.
  arXiv  Detail & Related papers  (2025-06-03T09:29:47Z)
• Exploring Representation-Aligned Latent Space for Better Generation [86.45670422239317]
  We introduce ReaLS, which integrates semantic priors to improve generation performance.<n>We show that fundamental DiT and SiT trained on ReaLS can achieve a 15% improvement in FID metric.<n>The enhanced semantic latent space enables more perceptual downstream tasks, such as segmentation and depth estimation.
  arXiv  Detail & Related papers  (2025-02-01T07:42:12Z)
• Mapping the Multiverse of Latent Representations [17.2089620240192]
  PRESTO is a principled framework for mapping the multiverse of machine-learning models that rely on latent representations. Our framework uses persistent homology to characterize the latent spaces arising from different combinations of diverse machine-learning methods.
  arXiv  Detail & Related papers  (2024-02-02T15:54:53Z)
• Learning Conditional Invariance through Cycle Consistency [60.85059977904014]
  We propose a novel approach to identify meaningful and independent factors of variation in a dataset. Our method involves two separate latent subspaces for the target property and the remaining input information. We demonstrate on synthetic and molecular data that our approach identifies more meaningful factors which lead to sparser and more interpretable models.
  arXiv  Detail & Related papers  (2021-11-25T17:33:12Z)
• Learning Disentangled Representations with Latent Variation Predictability [102.4163768995288]
  This paper defines the variation predictability of latent disentangled representations. Within an adversarial generation process, we encourage variation predictability by maximizing the mutual information between latent variations and corresponding image pairs. We develop an evaluation metric that does not rely on the ground-truth generative factors to measure the disentanglement of latent representations.
  arXiv  Detail & Related papers  (2020-07-25T08:54:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.