Bend the Basics: Degradation-Aware Deformable Tokenization for All-in-One Image Restoration

• URL: http://arxiv.org/abs/2608.06832v1
• Date: Fri, 07 Aug 2026 05:44:31 GMT
• Title: Bend the Basics: Degradation-Aware Deformable Tokenization for All-in-One Image Restoration
• Abstract summary: All-in-one image restoration seeks a single model that can recover images degraded by diverse and spatially non-uniform corruptions.<n>We present Flexible Image Transformer (FIT) that explicitly models degradation awareness across the entire pipeline.<n>FIT achieves state-of-the-art performance with 30.72 dB average PSNR on the five-degradation setting and 32.83 dB on the three-degradation setting, outperforming recent unified restoration methods by +0.5$sim$1.1 dB.
• Score: 72.64498743287601
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: All-in-one image restoration seeks a single model that can recover images degraded by diverse and spatially non-uniform corruptions. However, many unified Transformers rely on fixed patch partitioning: task/degradation condition is injected only into the backbone blocks after tokenization, leaving the embedding and reconstruction stages insensitive to local degradation variations. In contrast to previous approaches, we present Flexible Image Transformer (FIT) that explicitly models degradation awareness across the entire pipeline, from patch sampling to pixel reconstruction. Specifically, FIT employs a lightweight Degradation Encoder to predict a global degradation vector $\mathbf{g}$ and a spatial degradation map $\mathbf{M}$ from local degradation severity, which jointly condition the patch embedding and unembedding through adaptive deformation. Moreover, to improve robustness across degradation types, we introduce a task-token dropout strategy that regularizes task conditioning during training. On five standard benchmarks (BSD68, Rain100L, SOTS, GoPro, and LOLv1), FIT achieves state-of-the-art performance with 30.72 dB average PSNR on the five-degradation setting and 32.83 dB on the three-degradation setting, outperforming recent unified restoration methods by +0.5$\sim$1.1 dB. Moreover, the learned offsets provide a direct handle for visualizing degradation-aware spatial adaptation.

Related papers

• GraLoD: Graphics-Inspired Continuous Level-of-Detail Learning for Image Restoration [54.85451496831771]
  We propose GraLoD, a plug-and-play framework that treats restoration scale as a spatially varying and stage-dependent variable.<n>GraLoD reuses the native encoder hierarchy, aligns its multi-scale features into a shared LOD representation space, and predicts a stage-conditioned LOD field at each decoder stage.<n>Experiments demonstrate consistent improvements in task-specific and all-in-one restoration.
  arXiv  Detail & Related papers  (2026-09-15T03:22:22Z)
• Restoring Without Forgetting: Continual Learning Across Image Degradations [6.288045889067255]
  Recent progress in image restoration has converged on all-in-one architectures that jointly handle multiple degradations within a single network.<n>In practice, degradations are encountered sequentially as field-deployed systems progressively face new environmental conditions.<n>We formulate multi-degradation image restoration as a continual domain-incremental learning problem.
  arXiv  Detail & Related papers  (2026-08-24T20:04:18Z)
• What to Remove, What to Preserve: Dual-Ambiguity Rectification for All-in-One Image Restoration [22.756769058194788]
  We propose DAR-Net, a Dual-Ambiguity Rectification Network for all-in-one image restoration.<n>DAR-Net first introduces a structured degradation state through simplex-constrained archetype mixture modeling.<n>A Semantic Ambiguity Rectification (SeAR) module generates degradation-aware prompts to improve channel-wise conditioning in the decoder.
  arXiv  Detail & Related papers  (2026-07-30T17:01:24Z)
• DVANet: Degradation-aware Visual-prior Alignment Network for Image Restoration [31.441586261696617]
  This paper proposes DVANet, a collaborative unfolding process between degradation-aware observation consistency and visual-prior-guided reconstruction.<n>Experiments demonstrate that DVANet achieves superior or competitive performance on multi-scenario degradation and cross-domain image restoration tasks.
  arXiv  Detail & Related papers  (2026-06-17T14:09:39Z)
• Restoration-Aligned Generative Flow Models for Blind Motion Deblurring [57.425021485907756]
  We present DeFlow, a framework that resolves misalignment by reformulating the flow trajectory itself.<n>Under this formulation, the standard flow matching loss naturally takes the form of a residual loss.<n>Experiments on GoPro, HIDE, RealBlur, and RWBI demonstrate that DeFlow achieves strong restoration fidelity and perceptual realism.
  arXiv  Detail & Related papers  (2026-05-09T10:05:51Z)
• Learning to Restore Multi-Degraded Images via Ingredient Decoupling and Task-Aware Path Adaptation [51.10017611491389]
  Real-world images often suffer from multiple coexisting degradations, such as rain, noise, and haze coexisting in a single image.<n>We propose an adaptive multi-degradation image restoration network that reconstructs images by leveraging decoupled representations of degradation ingredients.<n>The resulting tightly integrated architecture, termed IMDNet, is extensively validated through experiments.
  arXiv  Detail & Related papers  (2025-11-07T01:50:36Z)
• Degradation-Aware All-in-One Image Restoration via Latent Prior Encoding [46.146259918814735]
  Real-world images often suffer from spatially diverse degradations such as haze, rain, snow, and low-light.<n>Existing all-in-one restoration approaches either depend on external text prompts or embed hand-crafted architectural priors.<n>We propose to reframe AIR as learned latent prior inference, where degradation-aware representations are automatically inferred from the input.
  arXiv  Detail & Related papers  (2025-09-22T13:51:09Z)
• AllRestorer: All-in-One Transformer for Image Restoration under Composite Degradations [52.076067325999226]
  We propose a novel Transformer-based restoration framework, AllRestorer. AllRestorer adaptively considers all image impairments, thereby avoiding errors from scene descriptor misdirection. We show that AllRestorer achieves a 5.00 dB increase in PSNR compared to the baseline on the CDD-11 dataset.
  arXiv  Detail & Related papers  (2024-11-16T05:30:55Z)
• Invertible Rescaling Network and Its Extensions [118.72015270085535]
  In this work, we propose a novel invertible framework to model the bidirectional degradation and restoration from a new perspective. We develop invertible models to generate valid degraded images and transform the distribution of lost contents. Then restoration is made tractable by applying the inverse transformation on the generated degraded image together with a randomly-drawn latent variable.
  arXiv  Detail & Related papers  (2022-10-09T06:58:58Z)
• Spatially-Adaptive Image Restoration using Distortion-Guided Networks [51.89245800461537]
  We present a learning-based solution for restoring images suffering from spatially-varying degradations. We propose SPAIR, a network design that harnesses distortion-localization information and dynamically adjusts to difficult regions in the image.
  arXiv  Detail & Related papers  (2021-08-19T11:02:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.