Geometric Autoencoder for Diffusion Models

• URL: http://arxiv.org/abs/2603.10365v2
• Date: Thu, 12 Mar 2026 12:00:55 GMT
• Title: Geometric Autoencoder for Diffusion Models
• Abstract summary: Latent diffusion models have established a new state-of-the-art in high-resolution visual generation.<n>We propose Geometric Autoencoder (GAE), a principled framework that unifies semantic discriminability, fidelity, and compactness.<n>GAE achieves compelling performance on the ImageNet-1K $256 times 256$ benchmark, reaching a gFID of 1.82 at only 80 epochs and 1.31 at 800 epochs.
• Score: 16.699694260098948
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Latent diffusion models have established a new state-of-the-art in high-resolution visual generation. Integrating Vision Foundation Model priors improves generative efficiency, yet existing latent designs remain largely heuristic. These approaches often struggle to unify semantic discriminability, reconstruction fidelity, and latent compactness. In this paper, we propose Geometric Autoencoder (GAE), a principled framework that systematically addresses these challenges. By analyzing various alignment paradigms, GAE constructs an optimized low-dimensional semantic supervision target from VFMs to provide guidance for the autoencoder. Furthermore, we leverage latent normalization that replaces the restrictive KL-divergence of standard VAEs, enabling a more stable latent manifold specifically optimized for diffusion learning. To ensure robust reconstruction under high-intensity noise, GAE incorporates a dynamic noise sampling mechanism. Empirically, GAE achieves compelling performance on the ImageNet-1K $256 \times 256$ benchmark, reaching a gFID of 1.82 at only 80 epochs and 1.31 at 800 epochs without Classifier-Free Guidance, significantly surpassing existing state-of-the-art methods. Beyond generative quality, GAE establishes a superior equilibrium between compression, semantic depth and robust reconstruction stability. These results validate our design considerations, offering a promising paradigm for latent diffusion modeling. Code and models are publicly available at https://github.com/sii-research/GAE.

Related papers

• Unifying Model-Free Efficiency and Model-Based Representations via Latent Dynamics [6.208369829942616]
  We present Unified Latent Dynamics (ULD), a novel reinforcement learning algorithm.<n>ULD unifies the efficiency of model-free methods with the representational strengths of model-based approaches.<n> evaluated on 80 environments spanning Gym locomotion, DeepMind Control (proprioceptive and visual), and Atari.
  arXiv  Detail & Related papers  (2026-02-13T06:06:56Z)
• Denoising Vision Transformer Autoencoder with Spectral Self-Regularization [21.85836384863372]
  We show that redundant high-frequency components in high-dimensional latent spaces hinder the training convergence of diffusion models.<n>We propose a spectral self-regularization strategy to suppress redundant high-frequency noise while simultaneously preserving reconstruction quality.<n>The resulting Denoising-VAE, a ViT-based autoencoder, produces cleaner, lower-noise latents, leading to improved generative quality and faster optimization convergence.
  arXiv  Detail & Related papers  (2025-11-16T15:00:32Z)
• From Structure to Detail: Hierarchical Distillation for Efficient Diffusion Model [18.782919607372328]
  Trajectory-based and distribution-based step distillation methods offer solutions.<n>Trajectory-based methods preserve global structure but act as a "lossy compressor"<n>We recast them into synergistic components within our novel Hierarchical Distillation framework.
  arXiv  Detail & Related papers  (2025-11-12T03:12:06Z)
• Vision Foundation Models Can Be Good Tokenizers for Latent Diffusion Models [45.63522160275318]
  Vision Foundation Model Variational Autoencoder (VFM-VAE) designed to resolve inherent tension between VFM's semantic focus and need for pixel-level fidelity.<n>Our system reaches a gFID (w/o CFG) of 2.20 in merely 80 epochs (a 10x speedup over prior tokenizers)
  arXiv  Detail & Related papers  (2025-10-21T09:30:45Z)
• Towards Efficient General Feature Prediction in Masked Skeleton Modeling [59.46799426434277]
  We propose a novel General Feature Prediction framework (GFP) for efficient mask skeleton modeling.<n>Our key innovation is replacing conventional low-level reconstruction with high-level feature prediction that spans from local motion patterns to global semantic representations.
  arXiv  Detail & Related papers  (2025-09-03T18:05:02Z)
• Vision Foundation Models as Effective Visual Tokenizers for Autoregressive Image Generation [52.261584726401686]
  We present a novel direction to build an image tokenizer directly on top of a frozen vision foundation model.<n>Based on these designs, our proposed image tokenizer, VFMTok, achieves substantial improvements in image reconstruction and generation quality.
  arXiv  Detail & Related papers  (2025-07-11T09:32:45Z)
• Masked Autoencoders Are Effective Tokenizers for Diffusion Models [56.08109308294133]
  MAETok is an autoencoder that learns semantically rich latent space while maintaining reconstruction fidelity.<n>MaETok achieves significant practical improvements, enabling a gFID of 1.69 with 76x faster training and 31x higher inference throughput for 512x512 generation.
  arXiv  Detail & Related papers  (2025-02-05T18:42:04Z)
• Exploring Representation-Aligned Latent Space for Better Generation [86.45670422239317]
  We introduce ReaLS, which integrates semantic priors to improve generation performance.<n>We show that fundamental DiT and SiT trained on ReaLS can achieve a 15% improvement in FID metric.<n>The enhanced semantic latent space enables more perceptual downstream tasks, such as segmentation and depth estimation.
  arXiv  Detail & Related papers  (2025-02-01T07:42:12Z)
• Beyond the Kolmogorov Barrier: A Learnable Weighted Hybrid Autoencoder for Model Order Reduction [0.8021197489470758]
  We propose a learnable weighted hybrid autoencoder to overcome the Kolmogorov barrier.<n>We empirically find that our trained model has a sharpness thousands of times smaller compared to other models.
  arXiv  Detail & Related papers  (2024-10-23T00:04:26Z)
• Model Inversion Attacks Through Target-Specific Conditional Diffusion Models [54.69008212790426]
  Model inversion attacks (MIAs) aim to reconstruct private images from a target classifier's training set, thereby raising privacy concerns in AI applications. Previous GAN-based MIAs tend to suffer from inferior generative fidelity due to GAN's inherent flaws and biased optimization within latent space. We propose Diffusion-based Model Inversion (Diff-MI) attacks to alleviate these issues.
  arXiv  Detail & Related papers  (2024-07-16T06:38:49Z)
• Conditional Denoising Diffusion for Sequential Recommendation [62.127862728308045]
  Two prominent generative models, Generative Adversarial Networks (GANs) and Variational AutoEncoders (VAEs) GANs suffer from unstable optimization, while VAEs are prone to posterior collapse and over-smoothed generations. We present a conditional denoising diffusion model, which includes a sequence encoder, a cross-attentive denoising decoder, and a step-wise diffuser.
  arXiv  Detail & Related papers  (2023-04-22T15:32:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.