LatentINDIGO: An INN-Guided Latent Diffusion Algorithm for Image Restoration

• URL: http://arxiv.org/abs/2505.12935v1
• Date: Mon, 19 May 2025 10:17:16 GMT
• Title: LatentINDIGO: An INN-Guided Latent Diffusion Algorithm for Image Restoration
• Authors: Di You, Daniel Siromani, Pier Luigi Dragotti,
• Abstract summary: We introduce a wavelet-inspired invertible neural network (INN) that simulates degradations through a forward transform and reconstructs lost details via the inverse transform.<n>Our algorithm achieves state-of-the-art performance on synthetic and real-world low-quality images, and can be readily adapted to arbitrary output sizes.
• Score: 19.74964267336191
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: There is a growing interest in the use of latent diffusion models (LDMs) for image restoration (IR) tasks due to their ability to model effectively the distribution of natural images. While significant progress has been made, there are still key challenges that need to be addressed. First, many approaches depend on a predefined degradation operator, making them ill-suited for complex or unknown degradations that deviate from standard analytical models. Second, many methods struggle to provide a stable guidance in the latent space and finally most methods convert latent representations back to the pixel domain for guidance at every sampling iteration, which significantly increases computational and memory overhead. To overcome these limitations, we introduce a wavelet-inspired invertible neural network (INN) that simulates degradations through a forward transform and reconstructs lost details via the inverse transform. We further integrate this design into a latent diffusion pipeline through two proposed approaches: LatentINDIGO-PixelINN, which operates in the pixel domain, and LatentINDIGO-LatentINN, which stays fully in the latent space to reduce complexity. Both approaches alternate between updating intermediate latent variables under the guidance of our INN and refining the INN forward model to handle unknown degradations. In addition, a regularization step preserves the proximity of latent variables to the natural image manifold. Experiments demonstrate that our algorithm achieves state-of-the-art performance on synthetic and real-world low-quality images, and can be readily adapted to arbitrary output sizes.

Related papers

• Rotation Equivariant Arbitrary-scale Image Super-Resolution [62.41329042683779]
  The arbitrary-scale image super-resolution (ASISR) aims to achieve arbitrary-scale high-resolution recoveries from a low-resolution input image.<n>We make efforts to construct a rotation equivariant ASISR method in this study.
  arXiv  Detail & Related papers  (2025-08-07T08:51:03Z)
• Pixel to Gaussian: Ultra-Fast Continuous Super-Resolution with 2D Gaussian Modeling [50.34513854725803]
  Arbitrary-scale super-resolution (ASSR) aims to reconstruct high-resolution (HR) images from low-resolution (LR) inputs with arbitrary upsampling factors.<n>We propose a novel ContinuousSR framework with a Pixel-to-Gaussian paradigm, which explicitly reconstructs 2D continuous HR signals from LR images using Gaussian Splatting.
  arXiv  Detail & Related papers  (2025-03-09T13:43:57Z)
• INDIGO+: A Unified INN-Guided Probabilistic Diffusion Algorithm for Blind and Non-Blind Image Restoration [22.19661915697775]
  We propose a novel INN-guided probabilistic diffusion algorithm for non-blind and blind image restoration.<n>INDIGO and BlindINDIGO combine the merits of the perfect reconstruction property of invertible neural networks (INN) with the strong generative capabilities of pre-trained diffusion models.
  arXiv  Detail & Related papers  (2025-01-23T18:51:52Z)
• Effective Diffusion Transformer Architecture for Image Super-Resolution [63.254644431016345]
  We design an effective diffusion transformer for image super-resolution (DiT-SR) In practice, DiT-SR leverages an overall U-shaped architecture, and adopts a uniform isotropic design for all the transformer blocks. We analyze the limitation of the widely used AdaLN, and present a frequency-adaptive time-step conditioning module.
  arXiv  Detail & Related papers  (2024-09-29T07:14:16Z)
• Binarized Diffusion Model for Image Super-Resolution [61.963833405167875]
  Binarization, an ultra-compression algorithm, offers the potential for effectively accelerating advanced diffusion models (DMs) Existing binarization methods result in significant performance degradation. We introduce a novel binarized diffusion model, BI-DiffSR, for image SR.
  arXiv  Detail & Related papers  (2024-06-09T10:30:25Z)
• Bidirectional Consistency Models [1.486435467709869]
  Diffusion models (DMs) are capable of generating remarkably high-quality samples by iteratively denoising a random vector.<n>DMs can also invert an input image to noise by moving backward along the probability flow ordinary differential equation (PF ODE)
  arXiv  Detail & Related papers  (2024-03-26T18:40:36Z)
• Distance Weighted Trans Network for Image Completion [52.318730994423106]
  We propose a new architecture that relies on Distance-based Weighted Transformer (DWT) to better understand the relationships between an image's components. CNNs are used to augment the local texture information of coarse priors. DWT blocks are used to recover certain coarse textures and coherent visual structures.
  arXiv  Detail & Related papers  (2023-10-11T12:46:11Z)
• Multi-Context Dual Hyper-Prior Neural Image Compression [10.349258638494137]
  We propose a Transformer-based nonlinear transform to efficiently capture both local and global information from the input image. We also introduce a novel entropy model that incorporates two different hyperpriors to model cross-channel and spatial dependencies of the latent representation. Our experiments show that our proposed framework performs better than the state-of-the-art methods in terms of rate-distortion performance.
  arXiv  Detail & Related papers  (2023-09-19T17:44:44Z)
• INDigo: An INN-Guided Probabilistic Diffusion Algorithm for Inverse Problems [31.693710075183844]
  We propose a method that combines invertible neural networks (INN) and diffusion models for general inverse problems. Specifically, we train the forward process of INN to simulate an arbitrary degradation process and use the inverse as a reconstruction process. Our algorithm effectively estimates the details lost in the degradation process and is no longer limited by the requirement of knowing the closed-form expression of the degradation model.
  arXiv  Detail & Related papers  (2023-06-05T15:14:47Z)
• Multilevel Diffusion: Infinite Dimensional Score-Based Diffusion Models for Image Generation [2.5556910002263984]
  Score-based diffusion models (SBDM) have emerged as state-of-the-art approaches for image generation. This paper develops SBDMs in the infinite-dimensional setting, that is, we model the training data as functions supported on a rectangular domain. We demonstrate how to overcome two shortcomings of current SBDM approaches in the infinite-dimensional setting.
  arXiv  Detail & Related papers  (2023-03-08T18:10:10Z)
• Deep network series for large-scale high-dynamic range imaging [2.3759432635713895]
  We propose a new approach for large-scale high-dynamic range computational imaging. Deep Neural Networks (DNNs) trained end-to-end can solve linear inverse imaging problems almost instantaneously. Alternative Plug-and-Play approaches have proven effective to address high-dynamic range challenges, but rely on highly iterative algorithms.
  arXiv  Detail & Related papers  (2022-10-28T11:13:41Z)
• Deep Variational Network Toward Blind Image Restoration [60.45350399661175]
  Blind image restoration is a common yet challenging problem in computer vision. We propose a novel blind image restoration method, aiming to integrate both the advantages of them. Experiments on two typical blind IR tasks, namely image denoising and super-resolution, demonstrate that the proposed method achieves superior performance over current state-of-the-arts.
  arXiv  Detail & Related papers  (2020-08-25T03:30:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.