Continual All-in-One Adverse Weather Removal with Knowledge Replay on a Unified Network Structure

• URL: http://arxiv.org/abs/2403.07292v1
• Date: Tue, 12 Mar 2024 03:50:57 GMT
• Title: Continual All-in-One Adverse Weather Removal with Knowledge Replay on a Unified Network Structure
• Authors: De Cheng, Yanling Ji, Dong Gong, Yan Li, Nannan Wang, Junwei Han, Dingwen Zhang
• Abstract summary: In real-world applications, image degeneration caused by adverse weather is always complex and changes with different weather conditions from days and seasons. We develop a novel continual learning framework with effective knowledge replay (KR) on a unified network structure. It considers the characteristics of the image restoration task with multiple degenerations in continual learning, and the knowledge for different degenerations can be shared and accumulated.
• Score: 92.8834309803903
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: In real-world applications, image degeneration caused by adverse weather is always complex and changes with different weather conditions from days and seasons. Systems in real-world environments constantly encounter adverse weather conditions that are not previously observed. Therefore, it practically requires adverse weather removal models to continually learn from incrementally collected data reflecting various degeneration types. Existing adverse weather removal approaches, for either single or multiple adverse weathers, are mainly designed for a static learning paradigm, which assumes that the data of all types of degenerations to handle can be finely collected at one time before a single-phase learning process. They thus cannot directly handle the incremental learning requirements. To address this issue, we made the earliest effort to investigate the continual all-in-one adverse weather removal task, in a setting closer to real-world applications. Specifically, we develop a novel continual learning framework with effective knowledge replay (KR) on a unified network structure. Equipped with a principal component projection and an effective knowledge distillation mechanism, the proposed KR techniques are tailored for the all-in-one weather removal task. It considers the characteristics of the image restoration task with multiple degenerations in continual learning, and the knowledge for different degenerations can be shared and accumulated in the unified network structure. Extensive experimental results demonstrate the effectiveness of the proposed method to deal with this challenging task, which performs competitively to existing dedicated or joint training image restoration methods. Our code is available at https://github.com/xiaojihh/CL_all-in-one.

Related papers

• SemiDDM-Weather: A Semi-supervised Learning Framework for All-in-one Adverse Weather Removal [57.52777076116241]
  Adverse weather removal aims to restore clear vision under adverse weather conditions. This paper presents a pioneering semi-supervised all-in-one adverse weather removal framework built on the teacher-student network.
  arXiv  Detail & Related papers  (2024-09-29T12:12:22Z)
• Teaching Tailored to Talent: Adverse Weather Restoration via Prompt Pool and Depth-Anything Constraint [15.733168323227174]
  We introduce a novel pipeline, T3-DiffWeather, to handle unpredictable weather input. We employ a prompt pool that allows the network to autonomously combine sub-prompts to construct weather-prompts. Our method achieves state-of-the-art performance across various synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2024-09-24T04:46:18Z)
• Genuine Knowledge from Practice: Diffusion Test-Time Adaptation for Video Adverse Weather Removal [53.15046196592023]
  We introduce test-time adaptation into adverse weather removal in videos. We propose the first framework that integrates test-time adaptation into the iterative diffusion reverse process.
  arXiv  Detail & Related papers  (2024-03-12T14:21:30Z)
• Language-driven All-in-one Adverse Weather Removal [11.47695460133523]
  All-in-one (AiO) frameworks restore various adverse weather degradations with a single set of networks jointly. Existing methods rely on extra supervision signals, which are usually unknown in real-world applications. We propose a Language-driven Restoration framework (LDR) to alleviate the aforementioned issues.
  arXiv  Detail & Related papers  (2023-12-03T13:05:54Z)
• Typhoon Intensity Prediction with Vision Transformer [51.84456610977905]
  We introduce "Typhoon Intensity Transformer" (Tint) to predict typhoon intensity accurately across space and time. Tint uses self-attention mechanisms with global receptive fields per layer. Experiments on a publicly available typhoon benchmark validate the efficacy of Tint.
  arXiv  Detail & Related papers  (2023-11-28T03:11:33Z)
• MetaWeather: Few-Shot Weather-Degraded Image Restoration [17.63266150036311]
  We introduce MetaWeather, a universal approach that can handle diverse and novel weather conditions with a single unified model. We show that MetaWeather can adapt to unseen weather conditions, significantly outperforming state-of-the-art multi-weather image restoration methods.
  arXiv  Detail & Related papers  (2023-08-28T06:25:40Z)
• Exploring the Application of Large-scale Pre-trained Models on Adverse Weather Removal [97.53040662243768]
  We propose a CLIP embedding module to make the network handle different weather conditions adaptively. This module integrates the sample specific weather prior extracted by CLIP image encoder together with the distribution specific information learned by a set of parameters.
  arXiv  Detail & Related papers  (2023-06-15T10:06:13Z)
• Unsupervised Restoration of Weather-affected Images using Deep Gaussian Process-based CycleGAN [92.15895515035795]
  We describe an approach for supervising deep networks that are based on CycleGAN. We introduce new losses for training CycleGAN that lead to more effective training, resulting in high-quality reconstructions. We demonstrate that the proposed method can be effectively applied to different restoration tasks like de-raining, de-hazing and de-snowing.
  arXiv  Detail & Related papers  (2022-04-23T01:30:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.