Always Clear Days: Degradation Type and Severity Aware All-In-One Adverse Weather Removal

• URL: http://arxiv.org/abs/2310.18293v2
• Date: Thu, 7 Mar 2024 08:49:17 GMT
• Title: Always Clear Days: Degradation Type and Severity Aware All-In-One Adverse Weather Removal
• Authors: Yu-Wei Chen, Soo-Chang Pei
• Abstract summary: All-in-one adverse weather removal is an emerging topic on image restoration, which aims to restore multiple weather degradations in an unified model. We propose a degradation type and severity aware model, called UtilityIR, for blind all-in-one bad weather image restoration.
• Score: 8.58670633761819
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: All-in-one adverse weather removal is an emerging topic on image restoration, which aims to restore multiple weather degradations in an unified model, and the challenge are twofold. First, discover and handle the property of multi-domain in target distribution formed by multiple weather conditions. Second, design efficient and effective operations for different degradations. To resolve this problem, most prior works focus on the multi-domain caused by different weather types. Inspired by inter\&intra-domain adaptation literature, we observe that not only weather type but also weather severity introduce multi-domain within each weather type domain, which is ignored by previous methods, and further limit their performance. To this end, we propose a degradation type and severity aware model, called UtilityIR, for blind all-in-one bad weather image restoration. To extract weather information from single image, we propose a novel Marginal Quality Ranking Loss (MQRL) and utilize Contrastive Loss (CL) to guide weather severity and type extraction, and leverage a bag of novel techniques such as Multi-Head Cross Attention (MHCA) and Local-Global Adaptive Instance Normalization (LG-AdaIN) to efficiently restore spatial varying weather degradation. The proposed method can outperform the state-of-the-art methods subjectively and objectively on different weather removal tasks with a large margin, and enjoy less model parameters. Proposed method even can restore unseen combined multiple degradation images, and modulate restoration level. Implementation code and pre-trained weights will be available at \url{https://github.com/fordevoted/UtilityIR}

Related papers

• Continual All-in-One Adverse Weather Removal with Knowledge Replay on a Unified Network Structure [92.8834309803903]
  In real-world applications, image degeneration caused by adverse weather is always complex and changes with different weather conditions from days and seasons. We develop a novel continual learning framework with effective knowledge replay (KR) on a unified network structure. It considers the characteristics of the image restoration task with multiple degenerations in continual learning, and the knowledge for different degenerations can be shared and accumulated.
  arXiv  Detail & Related papers  (2024-03-12T03:50:57Z)
• Decoupling Degradation and Content Processing for Adverse Weather Image Restoration [79.59228846484415]
  Adverse weather image restoration strives to recover clear images from those affected by various weather types, such as rain, haze, and snow. Previous techniques can handle multiple weather types within a single network, but they neglect the crucial distinction between these two processes, limiting the quality of restored images. This work introduces a novel adverse weather image restoration method, called DDCNet, which decouples the degradation removal and content reconstruction process at the feature level based on their channel statistics.
  arXiv  Detail & Related papers  (2023-12-08T12:26:38Z)
• Language-driven All-in-one Adverse Weather Removal [11.47695460133523]
  All-in-one (AiO) frameworks restore various adverse weather degradations with a single set of networks jointly. Existing methods rely on extra supervision signals, which are usually unknown in real-world applications. We propose a Language-driven Restoration framework (LDR) to alleviate the aforementioned issues.
  arXiv  Detail & Related papers  (2023-12-03T13:05:54Z)
• Cross-Consistent Deep Unfolding Network for Adaptive All-In-One Video Restoration [78.14941737723501]
  We propose a Cross-consistent Deep Unfolding Network (CDUN) for All-In-One VR. By orchestrating two cascading procedures, CDUN achieves adaptive processing for diverse degradations. In addition, we introduce a window-based inter-frame fusion strategy to utilize information from more adjacent frames.
  arXiv  Detail & Related papers  (2023-09-04T14:18:00Z)
• MetaWeather: Few-Shot Weather-Degraded Image Restoration [17.63266150036311]
  We introduce MetaWeather, a universal approach that can handle diverse and novel weather conditions with a single unified model. We show that MetaWeather can adapt to unseen weather conditions, significantly outperforming state-of-the-art multi-weather image restoration methods.
  arXiv  Detail & Related papers  (2023-08-28T06:25:40Z)
• Exploring the Application of Large-scale Pre-trained Models on Adverse Weather Removal [97.53040662243768]
  We propose a CLIP embedding module to make the network handle different weather conditions adaptively. This module integrates the sample specific weather prior extracted by CLIP image encoder together with the distribution specific information learned by a set of parameters.
  arXiv  Detail & Related papers  (2023-06-15T10:06:13Z)
• Counting Crowds in Bad Weather [68.50690406143173]
  We propose a method for robust crowd counting in adverse weather scenarios. Our model learns effective features and adaptive queries to account for large appearance variations. Experimental results show that the proposed algorithm is effective in counting crowds under different weather types on benchmark datasets.
  arXiv  Detail & Related papers  (2023-06-02T00:00:09Z)
• WM-MoE: Weather-aware Multi-scale Mixture-of-Experts for Blind Adverse Weather Removal [38.257012295118145]
  Mixture-of-Experts (MoE) is a popular model that adopts a learnable gate to route the input to different expert networks. We propose Weather-aware Multi-scale MoE (WM-MoE) based on Transformer for blind weather removal. Our method achieves state-of-the-art performance in blind adverse weather removal on two public datasets and our dataset.
  arXiv  Detail & Related papers  (2023-03-24T01:46:25Z)
• TransWeather: Transformer-based Restoration of Images Degraded by Adverse Weather Conditions [77.20136060506906]
  We propose TransWeather, a transformer-based end-to-end model with just a single encoder and a decoder. TransWeather achieves significant improvements across multiple test datasets over both All-in-One network. It is validated on real world test images and found to be more effective than previous methods.
  arXiv  Detail & Related papers  (2021-11-29T18:57:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.