Residual-Conditioned Optimal Transport: Towards Structure-Preserving Unpaired and Paired Image Restoration

• URL: http://arxiv.org/abs/2405.02843v2
• Date: Sat, 11 May 2024 02:30:06 GMT
• Title: Residual-Conditioned Optimal Transport: Towards Structure-Preserving Unpaired and Paired Image Restoration
• Authors: Xiaole Tang, Xin Hu, Xiang Gu, Jian Sun,
• Abstract summary: We propose a novel Residual-Conditioned Optimal Transport (RCOT) approach for image restoration. By duality, the RCOT problem is transformed into a minimax optimization problem, which can be solved by adversarially training neural networks.
• Score: 42.01716967725075
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep learning-based image restoration methods generally struggle with faithfully preserving the structures of the original image. In this work, we propose a novel Residual-Conditioned Optimal Transport (RCOT) approach, which models image restoration as an optimal transport (OT) problem for both unpaired and paired settings, introducing the transport residual as a unique degradation-specific cue for both the transport cost and the transport map. Specifically, we first formalize a Fourier residual-guided OT objective by incorporating the degradation-specific information of the residual into the transport cost. We further design the transport map as a two-pass RCOT map that comprises a base model and a refinement process, in which the transport residual is computed by the base model in the first pass and then encoded as a degradation-specific embedding to condition the second-pass restoration. By duality, the RCOT problem is transformed into a minimax optimization problem, which can be solved by adversarially training neural networks. Extensive experiments on multiple restoration tasks show that RCOT achieves competitive performance in terms of both distortion measures and perceptual quality, restoring images with more faithful structures as compared with state-of-the-art methods.

Related papers

• From Physical Degradation Models to Task-Aware All-in-One Image Restoration [44.45223512440674]
  All-in-one image restoration aims to adaptively handle multiple restoration tasks with a single trained model.<n>We adopt a physical degradation modeling perspective and predict a task-aware inverse degradation operator for efficient all-in-one image restoration.
  arXiv  Detail & Related papers  (2026-01-15T08:47:10Z)
• Reversible Inversion for Training-Free Exemplar-guided Image Editing [127.97756928865032]
  Existing approaches often require large-scale pre-training to learn relationships between the source and reference images.<n>Standard inversion is sub-optimal for EIE, leading to poor quality and inefficiency.<n>We introduce textbfReversible Inversion (ReInversion) for effective and efficient EIE.
  arXiv  Detail & Related papers  (2025-12-01T07:56:06Z)
• Physics-Guided Null-Space Diffusion with Sparse Masking for Corrective Sparse-View CT Reconstruction [5.479463752172751]
  Diffusion models have demonstrated remarkable generative capabilities in image processing tasks.<n>We propose a Sparse condition Temporal Rewighted Integrated Distribution Estimation guided diffusion model (STRIDE) for sparse-view CT reconstruction.<n> Experimental results on both public and real datasets demonstrate that the proposed method achieves the best improvement of 2.58 dB in PSNR, increase of 2.37% in SSIM, and reduction of 0.236 in MSE.
  arXiv  Detail & Related papers  (2025-09-07T09:42:16Z)
• Super-Resolution for Remote Sensing Imagery via the Coupling of a Variational Model and Deep Learning [20.697932997351813]
  gradient-guided multi-frame super-resolution (MFSR) framework for remote sensing imagery reconstruction. We propose a novel gradient-guided multi-frame super-resolution (MFSR) framework for remote sensing imagery reconstruction.
  arXiv  Detail & Related papers  (2024-12-13T04:19:48Z)
• Degradation-Aware Residual-Conditioned Optimal Transport for Unified Image Restoration [39.52747104610541]
  All-in-one image restoration has emerged as a practical and promising low-level vision task for real-world applications. We present a Degradation-Aware Residual-Conditioned Optimal Transport (DA-RCOT) approach that models (all-in-one) image restoration as an optimal transport problem. We show that DA-RCOT delivers superior adaptability to real-world scenarios even with multiple degradations and shows distinctive robustness to both degradation levels and the number of degradations.
  arXiv  Detail & Related papers  (2024-11-03T18:57:19Z)
• Unsupervised Low-dose CT Reconstruction with One-way Conditional Normalizing Flows [7.727981333251735]
  We propose a novel CNFs-based unsupervised LDCT iterative reconstruction algorithm. It employs strict one-way transformation when performing alternating optimization in the dual spaces. Experiments on different datasets suggest that the performance of the proposed algorithm could surpass some state-of-the-art unsupervised and even supervised methods.
  arXiv  Detail & Related papers  (2024-10-23T03:47:29Z)
• Learning Efficient and Effective Trajectories for Differential Equation-based Image Restoration [59.744840744491945]
  We reformulate the trajectory optimization of this kind of method, focusing on enhancing both reconstruction quality and efficiency. We propose cost-aware trajectory distillation to streamline complex paths into several manageable steps with adaptable sizes. Experiments showcase the significant superiority of the proposed method, achieving a maximum PSNR improvement of 2.1 dB over state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-10-07T07:46:08Z)
• One-step Generative Diffusion for Realistic Extreme Image Rescaling [47.89362819768323]
  We propose a novel framework called One-Step Image Rescaling Diffusion (OSIRDiff) for extreme image rescaling. OSIRDiff performs rescaling operations in the latent space of a pre-trained autoencoder. It effectively leverages powerful natural image priors learned by a pre-trained text-to-image diffusion model.
  arXiv  Detail & Related papers  (2024-08-17T09:51:42Z)
• SSP-IR: Semantic and Structure Priors for Diffusion-based Realistic Image Restoration [20.873676111265656]
  SSP-IR aims to fully exploit semantic and structure priors from low-quality images. Our method outperforms other state-of-the-art methods overall on both synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2024-07-04T04:55:14Z)
• Self-Supervised Coordinate Projection Network for Sparse-View Computed Tomography [31.774432128324385]
  We propose a Self-supervised COordinate Projection nEtwork (SCOPE) to reconstruct the artifacts-free CT image from a single SV sinogram. Compared with recent related works that solve similar problems using implicit neural representation network (INR), our essential contribution is an effective and simple re-projection strategy.
  arXiv  Detail & Related papers  (2022-09-12T06:14:04Z)
• Spatially-Adaptive Image Restoration using Distortion-Guided Networks [51.89245800461537]
  We present a learning-based solution for restoring images suffering from spatially-varying degradations. We propose SPAIR, a network design that harnesses distortion-localization information and dynamically adjusts to difficult regions in the image.
  arXiv  Detail & Related papers  (2021-08-19T11:02:25Z)
• Implicit Subspace Prior Learning for Dual-Blind Face Restoration [66.67059961379923]
  A novel implicit subspace prior learning (ISPL) framework is proposed as a generic solution to dual-blind face restoration. Experimental results demonstrate significant perception-distortion improvement of ISPL against existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-10-12T08:04:24Z)
• Structure-Preserving Super Resolution with Gradient Guidance [87.79271975960764]
  Structures matter in single image super resolution (SISR) Recent studies benefiting from generative adversarial network (GAN) have promoted the development of SISR. However, there are always undesired structural distortions in the recovered images.
  arXiv  Detail & Related papers  (2020-03-29T17:26:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.