Spatial Gram Alignment for Ultra-High-Resolution Image Synthesis

• URL: http://arxiv.org/abs/2605.20808v1
• Date: Wed, 20 May 2026 06:59:12 GMT
• Title: Spatial Gram Alignment for Ultra-High-Resolution Image Synthesis
• Abstract summary: Modern ultra-high-resolution image synthesis relies heavily on the robust generative capacity of large-scale pre-trained Latent Diffusion Models (LDMs)<n>We propose a novel framework that explicitly leverages the representation priors of vision foundation models while preserving the native generative capacity of LDMs.
• Score: 38.08544812796336
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Modern ultra-high-resolution image synthesis relies heavily on the robust generative capacity of large-scale pre-trained Latent Diffusion Models (LDMs). While recent representation alignment methods have proven effective by distilling visual priors from foundation models (e.g., SAM or DINO) into generative latent features, scaling these approaches to pre-trained LDMs at extreme resolutions exposes a critical learnability-fidelity conflict. Specifically, forcing direct patch-wise feature distillation inherently perturbs the pre-trained latent manifold, ultimately leading to generation degradation. To address this bottleneck, we propose Spatial Gram Alignment (SGA), a novel framework that explicitly leverages the representation priors of vision foundation models while preserving the native generative capacity of LDMs. Moving beyond restrictive direct alignment, SGA imposes a non-invasive spatial constraint by aligning the internal self-similarities of the generative features with those of the foundation priors. This spatial constraint effectively establishes macroscopic structural coherence, while the native generative objectives retain the microscopic pixel-level fidelity inherent to the original LDMs. Notably, this versatile strategy integrates seamlessly across both intermediate diffusion features and VAE latents within pre-trained LDMs. Extensive experiments demonstrate that SGA achieves state-of-the-art performance for ultra-high-resolution text-to-image synthesis, yielding an effective reconciliation between global structural integrity and fine-grained visual details. Code is available at https://github.com/zhang0jhon/SGA.

Related papers

• LPH-VTON: Resolving the Structure-Texture Dilemma of Virtual Try-On via Latent Process Handover [15.242291833451882]
  Virtual Try-On aims to synthesize photorealistic images of garments precisely aligned with a person's body and pose.<n>Current diffusion-based methods, however, face a fundamental trade-off between structural integrity and textural fidelity.<n>We propose LPH-VTON, a new synergistic framework that resolves this tension within a single, continuous denoising process.
  arXiv  Detail & Related papers  (2026-05-14T14:18:56Z)
• Deep Dreams Are Made of This: Visualizing Monosemantic Features in Diffusion Models [0.12277343096128711]
  This paper proposes latent visualization by optimization (LVO), a mechanistic interpretability technique that extends feature visualization by optimization to latent diffusion models.<n>LVO employs sparse autoencoders (SAEs) to disentangle polysemantic layer representations into monosemantic features.
  arXiv  Detail & Related papers  (2026-05-06T13:21:24Z)
• Coevolving Representations in Joint Image-Feature Diffusion [13.292849179066993]
  We argue that the representation space guiding diffusion should itself adapt to the generative task.<n>We propose Coevolving Diffusion Representation (CoReDi), a framework in which the semantic representation space evolves during training.<n>Experiments show that CoReDi achieves faster convergence and higher sample quality compared to joint diffusion models operating in fixed representation spaces.
  arXiv  Detail & Related papers  (2026-04-19T15:29:15Z)
• Better with Less: Tackling Heterogeneous Multi-Modal Image Joint Pretraining via Conditioned and Degraded Masked Autoencoder [34.73963627819185]
  High-resolution (HR) joint optical and synthetic aperture radar (SAR) pretraining seeks modality synergy to mutually enhance single-source representations.<n>We propose CoDe-MAE, pioneering a textitbetter synergy with less alignment philosophy.<n>CoDe-MAE successfully prevents representation degradation and establishes new state-of-the-art performance across diverse single- and bi-modal downstream tasks.
  arXiv  Detail & Related papers  (2026-04-18T10:23:00Z)
• SSG: Scaled Spatial Guidance for Multi-Scale Visual Autoregressive Generation [10.295970926059812]
  Visual autoregressive ( VAR) models generate images through next-scale prediction, naturally achieving coarse-to-fine, fast, high-fidelity mirroring human perception.<n>In practice, this hierarchy can drift at inference time, as limited capacity and accumulated error cause the model to deviate from its coarse-to-fine nature.<n>We propose Scaled Spatial Guidance (SSG), training-free, inference-time guidance that steers generation toward the intended hierarchy while maintaining global coherence.
  arXiv  Detail & Related papers  (2026-02-05T10:48:58Z)
• FUDOKI: Discrete Flow-based Unified Understanding and Generation via Kinetic-Optimal Velocities [76.46448367752944]
  multimodal large language models (MLLMs) unify visual understanding and image generation within a single framework.<n>Most existing MLLMs rely on autore (AR) architectures, which impose inherent limitations on future development.<n>We introduce FUDOKI, a unified multimodal model purely based on discrete flow matching.
  arXiv  Detail & Related papers  (2025-05-26T15:46:53Z)
• VRS-UIE: Value-Driven Reordering Scanning for Underwater Image Enhancement [104.78586859995333]
  State Space Models (SSMs) have emerged as a promising backbone for vision tasks due to their linear complexity and global receptive field.<n>The predominance of large-portion, homogeneous but useless oceanic backgrounds can dilute the feature representation responses of sparse yet valuable targets.<n>We propose a novel Value-Driven Reordering Scanning framework for Underwater Image Enhancement (UIE)<n>Our framework sets a new state-of-the-art, delivering superior enhancement performance (surpassing WMamba by 0.89 dB on average) by effectively suppressing water bias and preserving structural and color fidelity.
  arXiv  Detail & Related papers  (2025-05-02T12:21:44Z)
• Exploring Representation-Aligned Latent Space for Better Generation [86.45670422239317]
  We introduce ReaLS, which integrates semantic priors to improve generation performance.<n>We show that fundamental DiT and SiT trained on ReaLS can achieve a 15% improvement in FID metric.<n>The enhanced semantic latent space enables more perceptual downstream tasks, such as segmentation and depth estimation.
  arXiv  Detail & Related papers  (2025-02-01T07:42:12Z)
• JoReS-Diff: Joint Retinex and Semantic Priors in Diffusion Model for Low-light Image Enhancement [69.6035373784027]
  Low-light image enhancement (LLIE) has achieved promising performance by employing conditional diffusion models. Previous methods may neglect the importance of a sufficient formulation of task-specific condition strategy. We propose JoReS-Diff, a novel approach that incorporates Retinex- and semantic-based priors as the additional pre-processing condition.
  arXiv  Detail & Related papers  (2023-12-20T08:05:57Z)
• High-resolution Face Swapping via Latent Semantics Disentanglement [50.23624681222619]
  We present a novel high-resolution hallucination face swapping method using the inherent prior knowledge of a pre-trained GAN model. We explicitly disentangle the latent semantics by utilizing the progressive nature of the generator. We extend our method to video face swapping by enforcing two-temporal constraints on the latent space and the image space.
  arXiv  Detail & Related papers  (2022-03-30T00:33:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.