Degradation-Aware Residual-Conditioned Optimal Transport for Unified Image Restoration

• URL: http://arxiv.org/abs/2411.01656v1
• Date: Sun, 03 Nov 2024 18:57:19 GMT
• Title: Degradation-Aware Residual-Conditioned Optimal Transport for Unified Image Restoration
• Authors: Xiaole Tang, Xiang Gu, Xiaoyi He, Xin Hu, Jian Sun,
• Abstract summary: All-in-one image restoration has emerged as a practical and promising low-level vision task for real-world applications. We present a Degradation-Aware Residual-Conditioned Optimal Transport (DA-RCOT) approach that models (all-in-one) image restoration as an optimal transport problem. We show that DA-RCOT delivers superior adaptability to real-world scenarios even with multiple degradations and shows distinctive robustness to both degradation levels and the number of degradations.
• Score: 39.52747104610541
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: All-in-one image restoration has emerged as a practical and promising low-level vision task for real-world applications. In this context, the key issue lies in how to deal with different types of degraded images simultaneously. In this work, we present a Degradation-Aware Residual-Conditioned Optimal Transport (DA-RCOT) approach that models (all-in-one) image restoration as an optimal transport (OT) problem for unpaired and paired settings, introducing the transport residual as a degradation-specific cue for both the transport cost and the transport map. Specifically, we formalize image restoration with a residual-guided OT objective by exploiting the degradation-specific patterns of the Fourier residual in the transport cost. More crucially, we design the transport map for restoration as a two-pass DA-RCOT map, in which the transport residual is computed in the first pass and then encoded as multi-scale residual embeddings to condition the second-pass restoration. This conditioning process injects intrinsic degradation knowledge (e.g., degradation type and level) and structural information from the multi-scale residual embeddings into the OT map, which thereby can dynamically adjust its behaviors for all-in-one restoration. Extensive experiments across five degradations demonstrate the favorable performance of DA-RCOT as compared to state-of-the-art methods, in terms of distortion measures, perceptual quality, and image structure preservation. Notably, DA-RCOT delivers superior adaptability to real-world scenarios even with multiple degradations and shows distinctive robustness to both degradation levels and the number of degradations.

Related papers

• From Physical Degradation Models to Task-Aware All-in-One Image Restoration [44.45223512440674]
  All-in-one image restoration aims to adaptively handle multiple restoration tasks with a single trained model.<n>We adopt a physical degradation modeling perspective and predict a task-aware inverse degradation operator for efficient all-in-one image restoration.
  arXiv  Detail & Related papers  (2026-01-15T08:47:10Z)
• Physics-Guided Null-Space Diffusion with Sparse Masking for Corrective Sparse-View CT Reconstruction [5.479463752172751]
  Diffusion models have demonstrated remarkable generative capabilities in image processing tasks.<n>We propose a Sparse condition Temporal Rewighted Integrated Distribution Estimation guided diffusion model (STRIDE) for sparse-view CT reconstruction.<n> Experimental results on both public and real datasets demonstrate that the proposed method achieves the best improvement of 2.58 dB in PSNR, increase of 2.37% in SSIM, and reduction of 0.236 in MSE.
  arXiv  Detail & Related papers  (2025-09-07T09:42:16Z)
• DPMambaIR:All-in-One Image Restoration via Degradation-Aware Prompt State Space Model [36.979833523678614]
  All-in-One image restoration aims to address multiple image degradation problems. Existing approaches rely on Degradation-specific models or coarse-grained degradation prompts to guide image restoration. We propose DPMambaIR, a novel All-in-One image restoration framework.
  arXiv  Detail & Related papers  (2025-04-24T16:46:32Z)
• AllRestorer: All-in-One Transformer for Image Restoration under Composite Degradations [52.076067325999226]
  We propose a novel Transformer-based restoration framework, AllRestorer. AllRestorer adaptively considers all image impairments, thereby avoiding errors from scene descriptor misdirection. We show that AllRestorer achieves a 5.00 dB increase in PSNR compared to the baseline on the CDD-11 dataset.
  arXiv  Detail & Related papers  (2024-11-16T05:30:55Z)
• Perceive-IR: Learning to Perceive Degradation Better for All-in-One Image Restoration [33.163161549726446]
  Perceive-IR is an all-in-one image restorer designed to achieve fine-grained quality control. In the prompt learning stage, we leverage prompt learning to acquire a fine-grained quality perceiver capable of distinguishing three-tier quality levels. For the restoration stage, a semantic guidance module and compact feature extraction are proposed to further promote the restoration process.
  arXiv  Detail & Related papers  (2024-08-28T17:58:54Z)
• OneRestore: A Universal Restoration Framework for Composite Degradation [33.556183375565034]
  In real-world scenarios, image impairments often manifest as composite degradations, presenting a complex interplay of elements such as low light, haze, rain, and snow. Our study proposes a versatile imaging model that consolidates four physical corruption paradigms to accurately represent complex, composite degradation scenarios. OneRestore is a novel transformer-based framework designed for adaptive, controllable scene restoration.
  arXiv  Detail & Related papers  (2024-07-05T16:27:00Z)
• Residual-Conditioned Optimal Transport: Towards Structure-Preserving Unpaired and Paired Image Restoration [42.01716967725075]
  We propose a novel Residual-Conditioned Optimal Transport (RCOT) approach for image restoration. By duality, the RCOT problem is transformed into a minimax optimization problem, which can be solved by adversarially training neural networks.
  arXiv  Detail & Related papers  (2024-05-05T08:19:04Z)
• DR2: Diffusion-based Robust Degradation Remover for Blind Face Restoration [66.01846902242355]
  Blind face restoration usually synthesizes degraded low-quality data with a pre-defined degradation model for training. It is expensive and infeasible to include every type of degradation to cover real-world cases in the training data. We propose Robust Degradation Remover (DR2) to first transform the degraded image to a coarse but degradation-invariant prediction, then employ an enhancement module to restore the coarse prediction to a high-quality image.
  arXiv  Detail & Related papers  (2023-03-13T06:05:18Z)
• Invertible Rescaling Network and Its Extensions [118.72015270085535]
  In this work, we propose a novel invertible framework to model the bidirectional degradation and restoration from a new perspective. We develop invertible models to generate valid degraded images and transform the distribution of lost contents. Then restoration is made tractable by applying the inverse transformation on the generated degraded image together with a randomly-drawn latent variable.
  arXiv  Detail & Related papers  (2022-10-09T06:58:58Z)
• Spatially-Adaptive Image Restoration using Distortion-Guided Networks [51.89245800461537]
  We present a learning-based solution for restoring images suffering from spatially-varying degradations. We propose SPAIR, a network design that harnesses distortion-localization information and dynamically adjusts to difficult regions in the image.
  arXiv  Detail & Related papers  (2021-08-19T11:02:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.