Any Image Restoration via Efficient Spatial-Frequency Degradation Adaptation

• URL: http://arxiv.org/abs/2504.14249v1
• Date: Sat, 19 Apr 2025 09:54:46 GMT
• Title: Any Image Restoration via Efficient Spatial-Frequency Degradation Adaptation
• Authors: Bin Ren, Eduard Zamfir, Zongwei Wu, Yawei Li, Yidi Li, Danda Pani Paudel, Radu Timofte, Ming-Hsuan Yang, Luc Van Gool, Nicu Sebe,
• Abstract summary: Restoring any degraded image efficiently via just one model has become increasingly significant.<n>Our approach, termed AnyIR, takes a unified path that leverages inherent similarity across various degradations.<n>To fuse the degradation awareness and the contextualized attention, a spatial-frequency parallel fusion strategy is proposed.
• Score: 158.37640586809187
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Restoring any degraded image efficiently via just one model has become increasingly significant and impactful, especially with the proliferation of mobile devices. Traditional solutions typically involve training dedicated models per degradation, resulting in inefficiency and redundancy. More recent approaches either introduce additional modules to learn visual prompts, significantly increasing model size, or incorporate cross-modal transfer from large language models trained on vast datasets, adding complexity to the system architecture. In contrast, our approach, termed AnyIR, takes a unified path that leverages inherent similarity across various degradations to enable both efficient and comprehensive restoration through a joint embedding mechanism, without scaling up the model or relying on large language models.Specifically, we examine the sub-latent space of each input, identifying key components and reweighting them first in a gated manner. To fuse the intrinsic degradation awareness and the contextualized attention, a spatial-frequency parallel fusion strategy is proposed for enhancing spatial-aware local-global interactions and enriching the restoration details from the frequency perspective. Extensive benchmarking in the all-in-one restoration setting confirms AnyIR's SOTA performance, reducing model complexity by around 82\% in parameters and 85\% in FLOPs. Our code will be available at our Project page (https://amazingren.github.io/AnyIR/)

Related papers

• An Efficient and Mixed Heterogeneous Model for Image Restoration [71.85124734060665]
  Current mainstream approaches are based on three architectural paradigms: CNNs, Transformers, and Mambas.<n>We propose RestorMixer, an efficient and general-purpose IR model based on mixed-architecture fusion.
  arXiv  Detail & Related papers  (2025-04-15T08:19:12Z)
• Multi-dimensional Visual Prompt Enhanced Image Restoration via Mamba-Transformer Aggregation [4.227991281224256]
  This paper proposes to fully utilize complementary advantages from Mamba and Transformer without sacrificing computation efficiency.<n>The selective scanning mechanism of Mamba is employed to focus on spatial modeling, enabling capture long-range spatial dependencies.<n>The self-attention mechanism of Transformer is applied to focus on channel modeling, avoiding high burdens that are in quadratic growth with image's spatial dimensions.
  arXiv  Detail & Related papers  (2024-12-20T12:36:34Z)
• Hierarchical Information Flow for Generalized Efficient Image Restoration [108.83750852785582]
  We propose a hierarchical information flow mechanism for image restoration, dubbed Hi-IR. Hi-IR constructs a hierarchical information tree representing the degraded image across three levels. In seven common image restoration tasks, Hi-IR achieves its effectiveness and generalizability.
  arXiv  Detail & Related papers  (2024-11-27T18:30:08Z)
• Restore Anything Model via Efficient Degradation Adaptation [129.38475243424563]
  RAM takes a unified path that leverages inherent similarities across various degradations to enable efficient and comprehensive restoration.<n> RAM's SOTA performance confirms RAM's SOTA performance, reducing model complexity by approximately 82% in trainable parameters and 85% in FLOPs.
  arXiv  Detail & Related papers  (2024-07-18T10:26:53Z)
• Dynamic Pre-training: Towards Efficient and Scalable All-in-One Image Restoration [100.54419875604721]
  All-in-one image restoration tackles different types of degradations with a unified model instead of having task-specific, non-generic models for each degradation. We propose DyNet, a dynamic family of networks designed in an encoder-decoder style for all-in-one image restoration tasks. Our DyNet can seamlessly switch between its bulkier and lightweight variants, thereby offering flexibility for efficient model deployment.
  arXiv  Detail & Related papers  (2024-04-02T17:58:49Z)
• Look-Around Before You Leap: High-Frequency Injected Transformer for Image Restoration [46.96362010335177]
  In this paper, we propose HIT, a simple yet effective High-frequency Injected Transformer for image restoration. Specifically, we design a window-wise injection module (WIM), which incorporates abundant high-frequency details into the feature map, to provide reliable references for restoring high-quality images. In addition, we introduce a spatial enhancement unit (SEU) to preserve essential spatial relationships that may be lost due to the computations carried out across channel dimensions in the BIM.
  arXiv  Detail & Related papers  (2024-03-30T08:05:00Z)
• Multi-Stage Progressive Image Restoration [167.6852235432918]
  We propose a novel synergistic design that can optimally balance these competing goals. Our main proposal is a multi-stage architecture, that progressively learns restoration functions for the degraded inputs. The resulting tightly interlinked multi-stage architecture, named as MPRNet, delivers strong performance gains on ten datasets.
  arXiv  Detail & Related papers  (2021-02-04T18:57:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.