Fractal-IR: A Unified Framework for Efficient and Scalable Image Restoration

• URL: http://arxiv.org/abs/2503.17825v1
• Date: Sat, 22 Mar 2025 17:43:27 GMT
• Title: Fractal-IR: A Unified Framework for Efficient and Scalable Image Restoration
• Authors: Yawei Li, Bin Ren, Jingyun Liang, Rakesh Ranjan, Mengyuan Liu, Nicu Sebe, Ming-Hsuan Yang, Luca Benini,
• Abstract summary: We propose Fractal-IR, a fractal-based design that progressively refines degraded images by repeatedly expanding local information into broader regions.<n>This fractal architecture naturally captures local details at early stages and seamlessly transitions toward global context in deeper fractal stages.<n>We show that Fractal-IR achieves state-of-the-art performance in seven common image restoration tasks, including super-resolution, denoising, JPEG artifact removal, IR in adverse weather conditions, motion deblurring, defocus deblurring, and demosaicking.
• Score: 108.83750852785582
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: While vision transformers achieve significant breakthroughs in various image restoration (IR) tasks, it is still challenging to efficiently scale them across multiple types of degradations and resolutions. In this paper, we propose Fractal-IR, a fractal-based design that progressively refines degraded images by repeatedly expanding local information into broader regions. This fractal architecture naturally captures local details at early stages and seamlessly transitions toward global context in deeper fractal stages, removing the need for computationally heavy long-range self-attention mechanisms. Moveover, we observe the challenge in scaling up vision transformers for IR tasks. Through a series of analyses, we identify a holistic set of strategies to effectively guide model scaling. Extensive experimental results show that Fractal-IR achieves state-of-the-art performance in seven common image restoration tasks, including super-resolution, denoising, JPEG artifact removal, IR in adverse weather conditions, motion deblurring, defocus deblurring, and demosaicking. For $2\times$ SR on Manga109, Fractal-IR achieves a 0.21 dB PSNR gain. For grayscale image denoising on Urban100, Fractal-IR surpasses the previous method by 0.2 dB for $\sigma=50$.

Related papers

• Multi-Frame Blind Manifold Deconvolution for Rotating Synthetic Aperture Imaging [4.19203497706834]
  Rotating synthetic aperture (RSA) imaging system captures images of the target scene at different rotation angles by rotating a rectangular aperture.<n>Deblurring acquired RSA images plays a critical role in reconstructing a latent sharp image underlying the scene.<n>We propose a novel method to process RSA images using manifold fitting and penalisation in the content of blind convolution.
  arXiv  Detail & Related papers  (2025-01-31T18:39:47Z)
• UniUIR: Considering Underwater Image Restoration as An All-in-One Learner [49.35128836844725]
  We propose a Universal Underwater Image Restoration method, termed as UniUIR.<n>To decouple degradation-specific issues and explore the inter-correlations among various degradations in UIR task, we designed the Mamba Mixture-of-Experts module.<n>This module extracts degradation prior information in both spatial and frequency domains, and adaptively selects the most appropriate task-specific prompts.
  arXiv  Detail & Related papers  (2025-01-22T16:10:42Z)
• FCDM: A Physics-Guided Bidirectional Frequency Aware Convolution and Diffusion-Based Model for Sinogram Inpainting [14.043383277622874]
  We propose FCDM, a physics-guided, frequency-aware sinogram inpainting framework.<n>It integrates bidirectional frequency-domain convolutions to disentangle overlapping features while enforcing total absorption and frequency-domain consistency via a physics-informed loss.<n>Experiments on synthetic and real-world datasets show that FCDM outperforms existing methods, achieving SSIM over 0.95 and PSNR above 30 dB, with up to 33% and 29% improvements over baselines.
  arXiv  Detail & Related papers  (2024-08-26T12:31:38Z)
• Multi-Scale Representation Learning for Image Restoration with State-Space Model [13.622411683295686]
  We propose a novel Multi-Scale State-Space Model-based (MS-Mamba) for efficient image restoration. Our proposed method achieves new state-of-the-art performance while maintaining low computational complexity.
  arXiv  Detail & Related papers  (2024-08-19T16:42:58Z)
• Gated Multi-Resolution Transfer Network for Burst Restoration and Enhancement [75.25451566988565]
  We propose a novel Gated Multi-Resolution Transfer Network (GMTNet) to reconstruct a spatially precise high-quality image from a burst of low-quality raw images. Detailed experimental analysis on five datasets validates our approach and sets a state-of-the-art for burst super-resolution, burst denoising, and low-light burst enhancement.
  arXiv  Detail & Related papers  (2023-04-13T17:54:00Z)
• DR2: Diffusion-based Robust Degradation Remover for Blind Face Restoration [66.01846902242355]
  Blind face restoration usually synthesizes degraded low-quality data with a pre-defined degradation model for training. It is expensive and infeasible to include every type of degradation to cover real-world cases in the training data. We propose Robust Degradation Remover (DR2) to first transform the degraded image to a coarse but degradation-invariant prediction, then employ an enhancement module to restore the coarse prediction to a high-quality image.
  arXiv  Detail & Related papers  (2023-03-13T06:05:18Z)
• Spatially-Adaptive Image Restoration using Distortion-Guided Networks [51.89245800461537]
  We present a learning-based solution for restoring images suffering from spatially-varying degradations. We propose SPAIR, a network design that harnesses distortion-localization information and dynamically adjusts to difficult regions in the image.
  arXiv  Detail & Related papers  (2021-08-19T11:02:25Z)
• Learning to Restore a Single Face Image Degraded by Atmospheric Turbulence using CNNs [93.72048616001064]
  Images captured under such condition suffer from a combination of geometric deformation and space varying blur. We present a deep learning-based solution to the problem of restoring a turbulence-degraded face image.
  arXiv  Detail & Related papers  (2020-07-16T15:25:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.