PixRestore: Unified Image Restoration via Pixel Diffusion Transformer

• URL: http://arxiv.org/abs/2608.16793v1
• Date: Mon, 17 Aug 2026 16:49:55 GMT
• Title: PixRestore: Unified Image Restoration via Pixel Diffusion Transformer
• Abstract summary: Unified image restoration (UIR) aims to recover high-quality (HQ) content from low-quality (LQ) images with different degradations.<n>We present PixRestore, a VAE-free pixel-space Diffusion Transformer (DiT) for UIR, where the diffusion backbone is trained entirely from scratch.<n>PixRestore performs flow matching directly on patchified pixels, preserving fine-grained details while keeping the token sequence tractable.
• Score: 25.58481833908051
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Unified image restoration (UIR) aims to recover high-quality (HQ) content from low-quality (LQ) images with different degradations using a single model. Most recent methods adapt large pretrained text-to-image (T2I) latent diffusion models for their strong capacity and generative priors. However, the variational autoencoder (VAE) in latent T2I models may discard restoration-sensitive details, while the open-ended synthesis prior can introduce content-inconsistent artifacts. We present PixRestore, a VAE-free pixel-space Diffusion Transformer (DiT) for UIR, where the diffusion backbone is trained entirely from scratch, without relying on T2I pretraining. PixRestore performs flow matching directly on patchified pixels, preserving fine-grained details while keeping the token sequence tractable. To adapt to different degradations, PixRestore learns to predict the reliability of layer features using LQ--HQ DINO feature similarity. Features from more reliable layers are fused as dense conditioning, while less reliable layers receive stronger HQ-feature supervision to encourage degradation removal. We train PixRestore on a large-scale corpus of diverse scenes and degradations, and further finetune it into a one-step generator using DINO-based adversarial objectives for efficient inference. Experiments on public benchmarks and real-world test sets show that, with only about 50M parameters and single-step inference, PixRestore achieves the best overall fidelity, perceptual quality, and robustness to degradations among competing UIR models while being far more efficient. Larger PixRestore variants can further boost performance, demonstrating the scalability of our pixel-space design. Code and the curated benchmark can be found at https://github.com/csslc/PixRestore.

Related papers

• When Latents Forget Pixels: Restoring Fidelity in Diffusion Transformer Super-Resolution [47.89708788628452]
  We propose a pixel-grounded super-resolution (PGSR) framework that preserves LR-observed pixel evidence before VAE compression and reuses it throughout restoration.<n>Experiments demonstrate that PGSR improves the realism--fidelity trade-off and produces more faithful, visually convincing results than existing latent generative SR approaches.
  arXiv  Detail & Related papers  (2026-08-10T05:24:39Z)
• Eliminating VAE for Fast and High-Resolution Generative Detail Restoration [19.313842956605356]
  Diffusion models have attained remarkable breakthroughs in the real-world super-resolution (SR) task.<n>Recent works like GenDR adopt step distillation to minimize the step number to one.<n>GenDR-Pix can restore 4K image in only 1 second and 6GB.
  arXiv  Detail & Related papers  (2026-02-11T08:23:30Z)
• SSDD: Single-Step Diffusion Decoder for Efficient Image Tokenization [56.12853087022071]
  We introduce a new pixel diffusion decoder architecture for improved scaling and training stability.<n>We use distillation to replicate the performance of the diffusion decoder in an efficient single-step decoder.<n>This makes SSDD the first diffusion decoder optimized for single-step reconstruction trained without adversarial losses.
  arXiv  Detail & Related papers  (2025-10-06T15:57:31Z)
• Multi-scale Image Super Resolution with a Single Auto-Regressive Model [40.77470215283583]
  We tackle Image Super Resolution (ISR) using recent advances in Visual Auto-Regressive ( VAR) modeling.<n>To the best of our knowledge, this is the first time a quantizer is trained to force semantically consistent residuals at different scales.<n>Our model can denoise the LR image and super-resolve at half and full target upscale factors in a single forward pass.
  arXiv  Detail & Related papers  (2025-06-05T13:02:23Z)
• Timestep-Aware Diffusion Model for Extreme Image Rescaling [47.89362819768323]
  We propose a novel framework called Timestep-Aware Diffusion Model (TADM) for extreme image rescaling.<n>TADM performs rescaling operations in the latent space of a pre-trained autoencoder.<n>It effectively leverages powerful natural image priors learned by a pre-trained text-to-image diffusion model.
  arXiv  Detail & Related papers  (2024-08-17T09:51:42Z)
• Referring Flexible Image Restoration [9.664435854616633]
  We term this task Referring Flexible Image Restoration (RFIR) To address this, we first construct a large-scale synthetic dataset called RFIR, comprising 153,423 samples with the degraded image. We propose a novel transformer-based multi-task model named TransRFIR, which simultaneously perceives degradation types in the degraded image and removes specific degradation upon text prompt.
  arXiv  Detail & Related papers  (2024-04-16T07:25:17Z)
• Improving Diffusion-Based Image Synthesis with Context Prediction [49.186366441954846]
  Existing diffusion models mainly try to reconstruct input image from a corrupted one with a pixel-wise or feature-wise constraint along spatial axes. We propose ConPreDiff to improve diffusion-based image synthesis with context prediction. Our ConPreDiff consistently outperforms previous methods and achieves a new SOTA text-to-image generation results on MS-COCO, with a zero-shot FID score of 6.21.
  arXiv  Detail & Related papers  (2024-01-04T01:10:56Z)
• DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior [70.46245698746874]
  We present DiffBIR, a general restoration pipeline that could handle different blind image restoration tasks. DiffBIR decouples blind image restoration problem into two stages: 1) degradation removal: removing image-independent content; 2) information regeneration: generating the lost image content. In the first stage, we use restoration modules to remove degradations and obtain high-fidelity restored results. For the second stage, we propose IRControlNet that leverages the generative ability of latent diffusion models to generate realistic details.
  arXiv  Detail & Related papers  (2023-08-29T07:11:52Z)
• DR2: Diffusion-based Robust Degradation Remover for Blind Face Restoration [66.01846902242355]
  Blind face restoration usually synthesizes degraded low-quality data with a pre-defined degradation model for training. It is expensive and infeasible to include every type of degradation to cover real-world cases in the training data. We propose Robust Degradation Remover (DR2) to first transform the degraded image to a coarse but degradation-invariant prediction, then employ an enhancement module to restore the coarse prediction to a high-quality image.
  arXiv  Detail & Related papers  (2023-03-13T06:05:18Z)
• Rethinking the Paradigm of Content Constraints in Unpaired Image-to-Image Translation [9.900050049833986]
  We propose EnCo, a simple but efficient way to maintain the content by constraining the representational similarity in the latent space of patch-level features. For the similarity function, we use a simple MSE loss instead of contrastive loss, which is currently widely used in I2I tasks. In addition, we rethink the role played by discriminators in sampling patches and propose a discnative attention-guided (DAG) patch sampling strategy to replace random sampling.
  arXiv  Detail & Related papers  (2022-11-20T04:39:57Z)
• Restormer: Efficient Transformer for High-Resolution Image Restoration [118.9617735769827]
  convolutional neural networks (CNNs) perform well at learning generalizable image priors from large-scale data. Transformers have shown significant performance gains on natural language and high-level vision tasks. Our model, named Restoration Transformer (Restormer), achieves state-of-the-art results on several image restoration tasks.
  arXiv  Detail & Related papers  (2021-11-18T18:59:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.