Using Powerful Prior Knowledge of Diffusion Model in Deep Unfolding Networks for Image Compressive Sensing

• URL: http://arxiv.org/abs/2503.08429v1
• Date: Tue, 11 Mar 2025 13:40:46 GMT
• Title: Using Powerful Prior Knowledge of Diffusion Model in Deep Unfolding Networks for Image Compressive Sensing
• Authors: Chen Liao, Yan Shen, Dan Li, Zhongli Wang,
• Abstract summary: Deep Unfolding Networks (DUNs) have achieved impressive reconstruction quality in the field of image Compressive Sensing (CS)<n>We propose to use the powerful prior knowledge of pre-trained diffusion model in DUNs to achieve high-quality reconstruction with less steps for image CS.
• Score: 3.9833558904578075
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently, Deep Unfolding Networks (DUNs) have achieved impressive reconstruction quality in the field of image Compressive Sensing (CS) by unfolding iterative optimization algorithms into neural networks. The reconstruction quality of DUNs depends on the learned prior knowledge, so introducing stronger prior knowledge can further improve reconstruction quality. On the other hand, pre-trained diffusion models contain powerful prior knowledge and have a solid theoretical foundation and strong scalability, but it requires a large number of iterative steps to achieve reconstruction. In this paper, we propose to use the powerful prior knowledge of pre-trained diffusion model in DUNs to achieve high-quality reconstruction with less steps for image CS. Specifically, we first design an iterative optimization algorithm named Diffusion Message Passing (DMP), which embeds a pre-trained diffusion model into each iteration process of DMP. Then, we deeply unfold the DMP algorithm into a neural network named DMP-DUN. The proposed DMP-DUN can use lightweight neural networks to achieve mapping from measurement data to the intermediate steps of the reverse diffusion process and directly approximate the divergence of the diffusion model, thereby further improving reconstruction efficiency. Extensive experiments show that our proposed DMP-DUN achieves state-of-the-art performance and requires at least only 2 steps to reconstruct the image. Codes are available at https://github.com/FengodChen/DMP-DUN-CVPR2025.

Related papers

• Learning Efficient and Effective Trajectories for Differential Equation-based Image Restoration [59.744840744491945]
  We reformulate the trajectory optimization of this kind of method, focusing on enhancing both reconstruction quality and efficiency. We propose cost-aware trajectory distillation to streamline complex paths into several manageable steps with adaptable sizes. Experiments showcase the significant superiority of the proposed method, achieving a maximum PSNR improvement of 2.1 dB over state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-10-07T07:46:08Z)
• Effective Diffusion Transformer Architecture for Image Super-Resolution [63.254644431016345]
  We design an effective diffusion transformer for image super-resolution (DiT-SR) In practice, DiT-SR leverages an overall U-shaped architecture, and adopts a uniform isotropic design for all the transformer blocks. We analyze the limitation of the widely used AdaLN, and present a frequency-adaptive time-step conditioning module.
  arXiv  Detail & Related papers  (2024-09-29T07:14:16Z)
• Binarized Diffusion Model for Image Super-Resolution [61.963833405167875]
  Binarization, an ultra-compression algorithm, offers the potential for effectively accelerating advanced diffusion models (DMs) Existing binarization methods result in significant performance degradation. We introduce a novel binarized diffusion model, BI-DiffSR, for image SR.
  arXiv  Detail & Related papers  (2024-06-09T10:30:25Z)
• SDIP: Self-Reinforcement Deep Image Prior Framework for Image Processing [0.7673339435080445]
  Deep image prior (DIP) proposed in recent research has revealed the inherent trait of convolutional neural networks (CNN) for capturing substantial low-level image statistics priors. In this paper, we propose the self-reinforcement deep image prior (P) as an improved version of the original DIP.
  arXiv  Detail & Related papers  (2024-04-17T16:50:14Z)
• Invertible Diffusion Models for Compressed Sensing [22.293412255419614]
  Invertible Diffusion Models (IDM) is a novel efficient, end-to-end diffusion-based compressed sensing method.<n>Our IDM outperforms existing state-of-the-art CS networks by up to 2.64dB in PSNR.<n>Compared to the recent diffusion-based approach DDNM, our IDM achieves up to 10.09dB PSNR gain and 14.54 times faster inference.
  arXiv  Detail & Related papers  (2024-03-25T17:59:41Z)
• Graph Image Prior for Unsupervised Dynamic Cardiac Cine MRI Reconstruction [10.330083869344445]
  We propose a novel scheme for dynamic MRI representation, named Graph Image Prior'' (GIP) GIP adopts a two-stage generative network in a new modeling methodology, which first employs independent CNNs to recover the image structure for each frame. A graph convolutional network is utilized for feature fusion and image generation.
  arXiv  Detail & Related papers  (2024-03-23T08:57:46Z)
• MsDC-DEQ-Net: Deep Equilibrium Model (DEQ) with Multi-scale Dilated Convolution for Image Compressive Sensing (CS) [0.0]
  Compressive sensing (CS) is a technique that enables the recovery of sparse signals using fewer measurements than traditional sampling methods. We develop an interpretable and concise neural network model for reconstructing natural images using CS. The model, called MsDC-DEQ-Net, exhibits competitive performance compared to state-of-the-art network-based methods.
  arXiv  Detail & Related papers  (2024-01-05T16:25:58Z)
• Deep Multi-Threshold Spiking-UNet for Image Processing [51.88730892920031]
  This paper introduces the novel concept of Spiking-UNet for image processing, which combines the power of Spiking Neural Networks (SNNs) with the U-Net architecture. To achieve an efficient Spiking-UNet, we face two primary challenges: ensuring high-fidelity information propagation through the network via spikes and formulating an effective training strategy. Experimental results show that, on image segmentation and denoising, our Spiking-UNet achieves comparable performance to its non-spiking counterpart.
  arXiv  Detail & Related papers  (2023-07-20T16:00:19Z)
• Deep network series for large-scale high-dynamic range imaging [2.3759432635713895]
  We propose a new approach for large-scale high-dynamic range computational imaging. Deep Neural Networks (DNNs) trained end-to-end can solve linear inverse imaging problems almost instantaneously. Alternative Plug-and-Play approaches have proven effective to address high-dynamic range challenges, but rely on highly iterative algorithms.
  arXiv  Detail & Related papers  (2022-10-28T11:13:41Z)
• Neural Data-Dependent Transform for Learned Image Compression [72.86505042102155]
  We build a neural data-dependent transform and introduce a continuous online mode decision mechanism to jointly optimize the coding efficiency for each individual image. The experimental results show the effectiveness of the proposed neural-syntax design and the continuous online mode decision mechanism.
  arXiv  Detail & Related papers  (2022-03-09T14:56:48Z)
• Is Deep Image Prior in Need of a Good Education? [57.3399060347311]
  Deep image prior was introduced as an effective prior for image reconstruction. Despite its impressive reconstructive properties, the approach is slow when compared to learned or traditional reconstruction techniques. We develop a two-stage learning paradigm to address the computational challenge.
  arXiv  Detail & Related papers  (2021-11-23T15:08:26Z)
• Direct PET Image Reconstruction Incorporating Deep Image Prior and a Forward Projection Model [0.0]
  Convolutional neural networks (CNNs) have recently achieved remarkable performance in positron emission tomography (PET) image reconstruction. We propose an unsupervised direct PET image reconstruction method that incorporates a deep image prior framework. Our proposed method incorporates a forward projection model with a loss function to achieve unsupervised direct PET image reconstruction from sinograms.
  arXiv  Detail & Related papers  (2021-09-02T08:07:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.