Multi-Weather Image Restoration via Histogram-Based Transformer Feature Enhancement

• URL: http://arxiv.org/abs/2409.06334v1
• Date: Tue, 10 Sep 2024 08:47:03 GMT
• Title: Multi-Weather Image Restoration via Histogram-Based Transformer Feature Enhancement
• Authors: Yang Wen, Anyu Lai, Bo Qian, Hao Wang, Wuzhen Shi, Wenming Cao,
• Abstract summary: In adverse weather conditions, single-weather restoration models struggle to meet practical demands. There is an urgent need for a model capable of effectively handling mixed weather conditions and enhancing image quality in an automated manner. We propose a Task Sequence Generator module that, in conjunction with the Task Intra-patch Block, effectively extracts task-specific features embedded in degraded images.
• Score: 14.986500375481546
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Currently, the mainstream restoration tasks under adverse weather conditions have predominantly focused on single-weather scenarios. However, in reality, multiple weather conditions always coexist and their degree of mixing is usually unknown. Under such complex and diverse weather conditions, single-weather restoration models struggle to meet practical demands. This is particularly critical in fields such as autonomous driving, where there is an urgent need for a model capable of effectively handling mixed weather conditions and enhancing image quality in an automated manner. In this paper, we propose a Task Sequence Generator module that, in conjunction with the Task Intra-patch Block, effectively extracts task-specific features embedded in degraded images. The Task Intra-patch Block introduces an external learnable sequence that aids the network in capturing task-specific information. Additionally, we employ a histogram-based transformer module as the backbone of our network, enabling the capture of both global and local dynamic range features. Our proposed model achieves state-of-the-art performance on public datasets.

Related papers

• Multiple weather images restoration using the task transformer and adaptive mixup strategy [14.986500375481546]
  We introduce a novel multi-task severe weather removal model that can effectively handle complex weather conditions in an adaptive manner. Our model incorporates a weather task sequence generator, enabling the self-attention mechanism to selectively focus on features specific to different weather types. Our proposed model has achieved state-of-the-art performance on the publicly available dataset.
  arXiv  Detail & Related papers  (2024-09-05T04:55:40Z)
• Continual All-in-One Adverse Weather Removal with Knowledge Replay on a Unified Network Structure [92.8834309803903]
  In real-world applications, image degeneration caused by adverse weather is always complex and changes with different weather conditions from days and seasons. We develop a novel continual learning framework with effective knowledge replay (KR) on a unified network structure. It considers the characteristics of the image restoration task with multiple degenerations in continual learning, and the knowledge for different degenerations can be shared and accumulated.
  arXiv  Detail & Related papers  (2024-03-12T03:50:57Z)
• Exposure Bracketing is All You Need for Unifying Image Restoration and Enhancement Tasks [50.822601495422916]
  We propose to utilize exposure bracketing photography to unify image restoration and enhancement tasks. Due to the difficulty in collecting real-world pairs, we suggest a solution that first pre-trains the model with synthetic paired data. In particular, a temporally modulated recurrent network (TMRNet) and self-supervised adaptation method are proposed.
  arXiv  Detail & Related papers  (2024-01-01T14:14:35Z)
• Exploring the Application of Large-scale Pre-trained Models on Adverse Weather Removal [97.53040662243768]
  We propose a CLIP embedding module to make the network handle different weather conditions adaptively. This module integrates the sample specific weather prior extracted by CLIP image encoder together with the distribution specific information learned by a set of parameters.
  arXiv  Detail & Related papers  (2023-06-15T10:06:13Z)
• WM-MoE: Weather-aware Multi-scale Mixture-of-Experts for Blind Adverse Weather Removal [38.257012295118145]
  Mixture-of-Experts (MoE) is a popular model that adopts a learnable gate to route the input to different expert networks. We propose Weather-aware Multi-scale MoE (WM-MoE) based on Transformer for blind weather removal. Our method achieves state-of-the-art performance in blind adverse weather removal on two public datasets and our dataset.
  arXiv  Detail & Related papers  (2023-03-24T01:46:25Z)
• SHIFT: A Synthetic Driving Dataset for Continuous Multi-Task Domain Adaptation [152.60469768559878]
  SHIFT is the largest multi-task synthetic dataset for autonomous driving. It presents discrete and continuous shifts in cloudiness, rain and fog intensity, time of day, and vehicle and pedestrian density. Our dataset and benchmark toolkit are publicly available at www.vis.xyz/shift.
  arXiv  Detail & Related papers  (2022-06-16T17:59:52Z)
• An Efficient Domain-Incremental Learning Approach to Drive in All Weather Conditions [8.436505917796174]
  Deep neural networks enable impressive visual perception performance for autonomous driving. They are prone to forgetting previously learned information when adapting to different weather conditions. We propose DISC -- Domain Incremental through Statistical Correction -- a simple zero-forgetting approach which can incrementally learn new tasks.
  arXiv  Detail & Related papers  (2022-04-19T11:39:20Z)
• Multiple-environment Self-adaptive Network for Aerial-view Geo-localization [85.52750931345287]
  Aerial-view geo-localization tends to determine an unknown position through matching the drone-view image with the geo-tagged satellite-view image. We propose a Multiple-environment Self-adaptive Network (MuSe-Net) to adjust the domain shift caused by environmental changing. In particular, MuSe-Net employs a two-branch neural network containing one multiple-environment style extraction network and one self-adaptive feature extraction network.
  arXiv  Detail & Related papers  (2022-04-18T16:04:29Z)
• Fast Image-Anomaly Mitigation for Autonomous Mobile Robots [27.049498074025088]
  Camera anomalies like rain or dust can severelydegrade image quality and its related tasks. In this work we address this importantissue by implementing a pre-processing step that can effectivelymitigate such artifacts in a real-time fashion.
  arXiv  Detail & Related papers  (2021-09-04T15:39:42Z)
• Multi-Model Learning for Real-Time Automotive Semantic Foggy Scene Understanding via Domain Adaptation [17.530091734327296]
  We propose an efficient end-to-end automotive semantic scene understanding approach that is robust to foggy weather conditions. Our approach incorporates RGB colour, depth and luminance images via distinct encoders with dense connectivity. Our model achieves comparable performance to contemporary approaches at a fraction of the overall model complexity.
  arXiv  Detail & Related papers  (2020-12-09T21:04:05Z)
• Multi-Agent Routing Value Iteration Network [88.38796921838203]
  We propose a graph neural network based model that is able to perform multi-agent routing based on learned value in a sparsely connected graph. We show that our model trained with only two agents on graphs with a maximum of 25 nodes can easily generalize to situations with more agents and/or nodes.
  arXiv  Detail & Related papers  (2020-07-09T22:16:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.