Implicit Image-to-Image Schrodinger Bridge for Image Restoration

• URL: http://arxiv.org/abs/2403.06069v2
• Date: Fri, 27 Sep 2024 12:23:04 GMT
• Title: Implicit Image-to-Image Schrodinger Bridge for Image Restoration
• Authors: Yuang Wang, Siyeop Yoon, Pengfei Jin, Matthew Tivnan, Sifan Song, Zhennong Chen, Rui Hu, Li Zhang, Quanzheng Li, Zhiqiang Chen, Dufan Wu,
• Abstract summary: The Image-to-Image Schr"odinger Bridge (I$2$SB) presents a promising alternative by starting the generative process from corrupted images. We introduce the Implicit Image-to-Image Schr"odinger Bridge (I$3$SB) to further accelerate the generative process of I$2$SB.
• Score: 13.138398298354113
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Diffusion-based models are widely recognized for their effectiveness in image restoration tasks; however, their iterative denoising process, which begins from Gaussian noise, often results in slow inference speeds. The Image-to-Image Schr\"odinger Bridge (I$^2$SB) presents a promising alternative by starting the generative process from corrupted images and leveraging training techniques from score-based diffusion models. In this paper, we introduce the Implicit Image-to-Image Schr\"odinger Bridge (I$^3$SB) to further accelerate the generative process of I$^2$SB. I$^3$SB reconfigures the generative process into a non-Markovian framework by incorporating the initial corrupted image into each step, while ensuring that the marginal distribution aligns with that of I$^2$SB. This allows for the direct use of the pretrained network from I$^2$SB. Extensive experiments on natural images, human face images, and medical images validate the acceleration benefits of I$^3$SB. Compared to I$^2$SB, I$^3$SB achieves the same perceptual quality with fewer generative steps, while maintaining equal or improved fidelity to the ground truth.

Related papers

• Fast constrained sampling in pre-trained diffusion models [77.21486516041391]
  Diffusion models have dominated the field of large, generative image models. We propose an algorithm for fast-constrained sampling in large pre-trained diffusion models.
  arXiv  Detail & Related papers  (2024-10-24T14:52:38Z)
• Blind Image Restoration via Fast Diffusion Inversion [17.139433082780037]
  Blind Image Restoration via fast Diffusion (BIRD) is a blind IR method that jointly optimize for the degradation model parameters and the restored image. A key idea in our method is not to modify the reverse sampling, i.e., not to alter all the intermediate latents, once an initial noise is sampled. We experimentally validate BIRD on several image restoration tasks and show that it achieves state of the art performance on all of them.
  arXiv  Detail & Related papers  (2024-05-29T23:38:12Z)
• SwiftBrush: One-Step Text-to-Image Diffusion Model with Variational Score Distillation [1.5892730797514436]
  Text-to-image diffusion models often suffer from slow iterative sampling processes. We present a novel image-free distillation scheme named $textbfSwiftBrush$. SwiftBrush achieves an FID score of $textbf16.67$ and a CLIP score of $textbf0.29$ on the COCO-30K benchmark.
  arXiv  Detail & Related papers  (2023-12-08T18:44:09Z)
• Beyond First-Order Tweedie: Solving Inverse Problems using Latent Diffusion [41.758635460235716]
  We introduce Second-order Tweedie sampler from Surrogate Loss (STSL) STSL offers efficiency comparable to first-order Tweedie with a tractable reverse process using second-order approximation. Our method surpasses SoTA solvers PSLD and P2L, achieving 4X and 8X reduction in neural function evaluations.
  arXiv  Detail & Related papers  (2023-12-01T14:36:24Z)
• ACT-Diffusion: Efficient Adversarial Consistency Training for One-step Diffusion Models [59.90959789767886]
  We show that optimizing consistency training loss minimizes the Wasserstein distance between target and generated distributions. By incorporating a discriminator into the consistency training framework, our method achieves improved FID scores on CIFAR10 and ImageNet 64$times$64 and LSUN Cat 256$times$256 datasets.
  arXiv  Detail & Related papers  (2023-11-23T16:49:06Z)
• Improving Denoising Diffusion Models via Simultaneous Estimation of Image and Noise [15.702941058218196]
  This paper introduces two key contributions aimed at improving the speed and quality of images generated through inverse diffusion processes. The first contribution involves re parameterizing the diffusion process in terms of the angle on a quarter-circular arc between the image and noise. The second contribution is to directly estimate both the image ($mathbfx_0$) and noise ($mathbfepsilon$) using our network.
  arXiv  Detail & Related papers  (2023-10-26T05:43:07Z)
• EGC: Image Generation and Classification via a Diffusion Energy-Based Model [59.591755258395594]
  This work introduces an energy-based classifier and generator, namely EGC, which can achieve superior performance in both tasks using a single neural network. EGC achieves competitive generation results compared with state-of-the-art approaches on ImageNet-1k, CelebA-HQ and LSUN Church. This work represents the first successful attempt to simultaneously excel in both tasks using a single set of network parameters.
  arXiv  Detail & Related papers  (2023-04-04T17:59:14Z)
• I$^2$SB: Image-to-Image Schr\"odinger Bridge [87.43524087956457]
  Image-to-Image Schr"odinger Bridge (I$2$SB) is a new class of conditional diffusion models. I$2$SB directly learns the nonlinear diffusion processes between two given distributions. We show that I$2$SB surpasses standard conditional diffusion models with more interpretable generative processes.
  arXiv  Detail & Related papers  (2023-02-12T08:35:39Z)
• Unsupervised Single Image Super-resolution Under Complex Noise [60.566471567837574]
  This paper proposes a model-based unsupervised SISR method to deal with the general SISR task with unknown degradations. The proposed method can evidently surpass the current state of the art (SotA) method (about 1dB PSNR) not only with a slighter model (0.34M vs. 2.40M) but also faster speed.
  arXiv  Detail & Related papers  (2021-07-02T11:55:40Z)
• Deep Variational Network Toward Blind Image Restoration [60.45350399661175]
  Blind image restoration is a common yet challenging problem in computer vision. We propose a novel blind image restoration method, aiming to integrate both the advantages of them. Experiments on two typical blind IR tasks, namely image denoising and super-resolution, demonstrate that the proposed method achieves superior performance over current state-of-the-arts.
  arXiv  Detail & Related papers  (2020-08-25T03:30:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.