When Latents Forget Pixels: Restoring Fidelity in Diffusion Transformer Super-Resolution

• URL: http://arxiv.org/abs/2608.09133v1
• Date: Mon, 10 Aug 2026 05:24:39 GMT
• Title: When Latents Forget Pixels: Restoring Fidelity in Diffusion Transformer Super-Resolution
• Abstract summary: We propose a pixel-grounded super-resolution (PGSR) framework that preserves LR-observed pixel evidence before VAE compression and reuses it throughout restoration.<n>Experiments demonstrate that PGSR improves the realism--fidelity trade-off and produces more faithful, visually convincing results than existing latent generative SR approaches.
• Score: 47.89708788628452
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Image super-resolution (SR) with large generative models has recently achieved remarkable perceptual quality, yet maintaining fidelity to the LR observation remains challenging. In particular, we observe that diffusion transformers (DiTs) built on latent representations suffer from a critical limitation: the compression bottleneck of the VAE weakens fine-grained spatial information, leading to hallucinated details that are weakly grounded in the input image. In this work, we revisit generative SR from a representation perspective and propose a pixel-grounded super-resolution (PGSR) framework that preserves LR-observed pixel evidence before VAE compression and reuses it throughout restoration. Instead of relying solely on the compressed latent condition, PGSR extracts pre-VAE pixel evidence from the upsampled LR image and reuses it at two stages. First, Condition-Side Trajectory Guidance fuses LR-derived pixel evidence with the latent LR condition to guide the latent restoration trajectory. Second, Decoder-Side Pixel Grounding injects multi-scale pixel features into the frozen VAE decoder to ground the final rendering with LR-observed cues. To efficiently adapt large pretrained DiT models, we keep the latent autoencoder and main flow-matching backbone frozen, and train only lightweight restoration modules. We further study an efficient local-window attention variant for improved high-resolution efficiency and scalability. Extensive experiments demonstrate that PGSR improves the realism--fidelity trade-off and produces more faithful, visually convincing results than existing latent generative SR approaches.

Related papers

• DNF-SR: Dual-Input and Negative-Aware Feature Fine-Tuning for Real-World Image Super-Resolution [50.847827209157494]
  Unmediatedly feeding LR into a diffusion model creates a distributional gap with the model's original input.<n>We propose a Dual-input and Negative-aware Feature fine-tuning method for Real-ISR.<n>We exploit this property and propose a post-training optimization method, Negative-aware Feature Fine-Tuning (NF2T), which guides the model toward producing higher-quality results.
  arXiv  Detail & Related papers  (2026-09-14T06:53:22Z)
• PixelIR: Fidelity-Perception Decoupling via Pixel-Space Image-Residual Flow Matching for Efficient One-Step Real-World Super-Resolution [28.787313933850687]
  Real-world image super-resolution (Real-ISR) aims to preserve structures supported by the degraded observation while reconstructing perceptually realistic details.<n>We propose PixelIR, a fidelity-perception decoupling framework built upon pixel-space image-residual flow matching.<n>Experiments show that PixelIR achieves leading PSNR, SSIM, and LPIPS on both RealSR and DRealSR.
  arXiv  Detail & Related papers  (2026-08-31T13:39:54Z)
• Ultra-High-Definition Restoration Transformers with Correlation Matching Transformation [113.23094779244771]
  UHDformer++ is a Transformer-based framework to solve Ultra-High-Definition (UHD) image restoration tasks.<n>It operates across four learning spaces: 1) a high-resolution space (HR) for multi-level feature extraction, 2) a low-resolution space (LR) for learning compact, representative features, 3) a super-resolution space (SR) for up low-resolution features from SR, and 4) a low-high fusion and reconstruction space (LHFR) for final image restoration.
  arXiv  Detail & Related papers  (2026-08-18T20:41:27Z)
• VARestorer: One-Step VAR Distillation for Real-World Image Super-Resolution [76.19751531910039]
  We propose VARestorer, a framework that transforms a pre-trained text-to-image VAR model into a one-step ISR model.<n>VARestorer achieves state-of-the-art performance with 72.32 MUSIQ and 0.7669 CLIPIQA on DIV2K dataset.
  arXiv  Detail & Related papers  (2026-04-23T09:09:36Z)
• Latent Diffusion, Implicit Amplification: Efficient Continuous-Scale Super-Resolution for Remote Sensing Images [7.920423405957888]
  E$2$DiffSR achieves superior objective metrics and visual quality compared to the state-of-the-art SR methods. It reduces the inference time of diffusion-based SR methods to a level comparable to that of non-diffusion methods.
  arXiv  Detail & Related papers  (2024-10-30T09:14:13Z)
• ControlSR: Taming Diffusion Models for Consistent Real-World Image Super Resolution [68.72454974431749]
  We present ControlSR, a new method that can tame Diffusion Models for consistent real-world image super-resolution (Real-ISR)<n>Our model can achieve better performance across multiple metrics on several test sets and generate more consistent SR results with LR images than existing methods.
  arXiv  Detail & Related papers  (2024-10-18T08:35:57Z)
• LaRE$^2$: Latent Reconstruction Error Based Method for Diffusion-Generated Image Detection [31.009153793849183]
  We come up with the Latent Reconstruction Error (LaRE), the first reconstruction-error based feature in the latent space for generated image detection.<n>To exploit LaRE, we propose an Error-Guided feature REfinement module (EGRE), which can refine the image feature guided by LaRE to enhance the discriminativeness of the feature.<n>Our EGRE utilizes an align-then-refine mechanism, which effectively refines the image feature for generated-image detection from both spatial and channel perspectives.
  arXiv  Detail & Related papers  (2024-03-26T07:55:16Z)
• Reti-Diff: Illumination Degradation Image Restoration with Retinex-based Latent Diffusion Model [59.08821399652483]
  Illumination degradation image restoration (IDIR) techniques aim to improve the visibility of degraded images and mitigate the adverse effects of deteriorated illumination. Among these algorithms, diffusion model (DM)-based methods have shown promising performance but are often burdened by heavy computational demands and pixel misalignment issues when predicting the image-level distribution. We propose to leverage DM within a compact latent space to generate concise guidance priors and introduce a novel solution called Reti-Diff for the IDIR task. Reti-Diff comprises two key components: the Retinex-based latent DM (RLDM) and the Retinex-guided transformer (RG
  arXiv  Detail & Related papers  (2023-11-20T09:55:06Z)
• SRDiff: Single Image Super-Resolution with Diffusion Probabilistic Models [19.17571465274627]
  Single image super-resolution (SISR) aims to reconstruct high-resolution (HR) images from the given low-resolution (LR) ones. We propose a novel single image super-resolution diffusion probabilistic model (SRDiff) SRDiff is optimized with a variant of the variational bound on the data likelihood and can provide diverse and realistic SR predictions.
  arXiv  Detail & Related papers  (2021-04-30T12:31:25Z)
• Characteristic Regularisation for Super-Resolving Face Images [81.84939112201377]
  Existing facial image super-resolution (SR) methods focus mostly on improving artificially down-sampled low-resolution (LR) imagery. Previous unsupervised domain adaptation (UDA) methods address this issue by training a model using unpaired genuine LR and HR data. This renders the model overstretched with two tasks: consistifying the visual characteristics and enhancing the image resolution. We formulate a method that joins the advantages of conventional SR and UDA models.
  arXiv  Detail & Related papers  (2019-12-30T16:27:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.