Prompt to Restore, Restore to Prompt: Cyclic Prompting for Universal Adverse Weather Removal

• URL: http://arxiv.org/abs/2503.09013v1
• Date: Wed, 12 Mar 2025 03:03:06 GMT
• Title: Prompt to Restore, Restore to Prompt: Cyclic Prompting for Universal Adverse Weather Removal
• Authors: Rongxin Liao, Feng Li, Yanyan Wei, Zenglin Shi, Le Zhang, Huihui Bai, Meng Wang,
• Abstract summary: Universal adverse weather removal (UAWR) seeks to address various weather degradations within a unified framework.<n>Recent methods are inspired by prompt learning using pre-trained vision-language models (e.g., CLIP)<n>We propose CyclicPrompt, an innovative cyclic prompt approach designed to enhance the effectiveness, adaptability, and generalizability of UAWR.
• Score: 19.896064731182985
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Universal adverse weather removal (UAWR) seeks to address various weather degradations within a unified framework. Recent methods are inspired by prompt learning using pre-trained vision-language models (e.g., CLIP), leveraging degradation-aware prompts to facilitate weather-free image restoration, yielding significant improvements. In this work, we propose CyclicPrompt, an innovative cyclic prompt approach designed to enhance the effectiveness, adaptability, and generalizability of UAWR. CyclicPrompt Comprises two key components: 1) a composite context prompt that integrates weather-related information and context-aware representations into the network to guide restoration. This prompt differs from previous methods by marrying learnable input-conditional vectors with weather-specific knowledge, thereby improving adaptability across various degradations. 2) The erase-and-paste mechanism, after the initial guided restoration, substitutes weather-specific knowledge with constrained restoration priors, inducing high-quality weather-free concepts into the composite prompt to further fine-tune the restoration process. Therefore, we can form a cyclic "Prompt-Restore-Prompt" pipeline that adeptly harnesses weather-specific knowledge, textual contexts, and reliable textures. Extensive experiments on synthetic and real-world datasets validate the superior performance of CyclicPrompt. The code is available at: https://github.com/RongxinL/CyclicPrompt.

Related papers

• Beyond Degradation Redundancy: Contrastive Prompt Learning for All-in-One Image Restoration [109.38288333994407]
  Contrastive Prompt Learning (CPL) is a novel framework that fundamentally enhances prompt-task alignment. Our framework establishes new state-of-the-art performance while maintaining parameter efficiency, offering a principled solution for unified image restoration.
  arXiv  Detail & Related papers  (2025-04-14T08:24:57Z)
• DA2Diff: Exploring Degradation-aware Adaptive Diffusion Priors for All-in-One Weather Restoration [32.16602874389847]
  We propose an innovative diffusion paradigm with degradation-aware adaptive priors for all-in-one weather restoration, termed DA2Diff. We deploy a set of learnable prompts to capture degradation-aware representations by the prompt-image similarity constraints in the CLIP space. We propose a dynamic expert selection modulator that employs a dynamic weather-aware router to flexibly dispatch varying numbers of restoration experts for each weather-distorted image.
  arXiv  Detail & Related papers  (2025-04-07T14:38:57Z)
• Teaching Tailored to Talent: Adverse Weather Restoration via Prompt Pool and Depth-Anything Constraint [15.733168323227174]
  We introduce a novel pipeline, T3-DiffWeather, to handle unpredictable weather input. We employ a prompt pool that allows the network to autonomously combine sub-prompts to construct weather-prompts. Our method achieves state-of-the-art performance across various synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2024-09-24T04:46:18Z)
• AWRaCLe: All-Weather Image Restoration using Visual In-Context Learning [29.309503214127016]
  All-Weather Image Restoration (AWIR) under adverse weather conditions is a challenging task due to the presence of different types of degradations.<n>We propose AWRaCLe, a novel approach for AWIR that innovatively utilizes degradation-specific visual context information to steer the image restoration process.<n>AWRaCLe incorporates Degradation Context Extraction (DCE) and Context Fusion (CF) to seamlessly integrate degradation-specific features from the context into an image restoration network.
  arXiv  Detail & Related papers  (2024-08-30T21:35:25Z)
• Continual All-in-One Adverse Weather Removal with Knowledge Replay on a Unified Network Structure [92.8834309803903]
  In real-world applications, image degeneration caused by adverse weather is always complex and changes with different weather conditions from days and seasons. We develop a novel continual learning framework with effective knowledge replay (KR) on a unified network structure. It considers the characteristics of the image restoration task with multiple degenerations in continual learning, and the knowledge for different degenerations can be shared and accumulated.
  arXiv  Detail & Related papers  (2024-03-12T03:50:57Z)
• SPIRE: Semantic Prompt-Driven Image Restoration [66.26165625929747]
  We develop SPIRE, a Semantic and restoration Prompt-driven Image Restoration framework. Our approach is the first framework that supports fine-level instruction through language-based quantitative specification of the restoration strength. Our experiments demonstrate the superior restoration performance of SPIRE compared to the state of the arts.
  arXiv  Detail & Related papers  (2023-12-18T17:02:30Z)
• Decoupling Degradation and Content Processing for Adverse Weather Image Restoration [79.59228846484415]
  Adverse weather image restoration strives to recover clear images from those affected by various weather types, such as rain, haze, and snow. Previous techniques can handle multiple weather types within a single network, but they neglect the crucial distinction between these two processes, limiting the quality of restored images. This work introduces a novel adverse weather image restoration method, called DDCNet, which decouples the degradation removal and content reconstruction process at the feature level based on their channel statistics.
  arXiv  Detail & Related papers  (2023-12-08T12:26:38Z)
• Language-driven All-in-one Adverse Weather Removal [11.47695460133523]
  All-in-one (AiO) frameworks restore various adverse weather degradations with a single set of networks jointly. Existing methods rely on extra supervision signals, which are usually unknown in real-world applications. We propose a Language-driven Restoration framework (LDR) to alleviate the aforementioned issues.
  arXiv  Detail & Related papers  (2023-12-03T13:05:54Z)
• PromptIR: Prompting for All-in-One Blind Image Restoration [64.02374293256001]
  We present a prompt-based learning approach, PromptIR, for All-In-One image restoration. Our method uses prompts to encode degradation-specific information, which is then used to dynamically guide the restoration network. PromptIR offers a generic and efficient plugin module with few lightweight prompts.
  arXiv  Detail & Related papers  (2023-06-22T17:59:52Z)
• Exploring the Application of Large-scale Pre-trained Models on Adverse Weather Removal [97.53040662243768]
  We propose a CLIP embedding module to make the network handle different weather conditions adaptively. This module integrates the sample specific weather prior extracted by CLIP image encoder together with the distribution specific information learned by a set of parameters.
  arXiv  Detail & Related papers  (2023-06-15T10:06:13Z)
• Unsupervised Restoration of Weather-affected Images using Deep Gaussian Process-based CycleGAN [92.15895515035795]
  We describe an approach for supervising deep networks that are based on CycleGAN. We introduce new losses for training CycleGAN that lead to more effective training, resulting in high-quality reconstructions. We demonstrate that the proposed method can be effectively applied to different restoration tasks like de-raining, de-hazing and de-snowing.
  arXiv  Detail & Related papers  (2022-04-23T01:30:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.