NightHazeFormer: Single Nighttime Haze Removal Using Prior Query Transformer

• URL: http://arxiv.org/abs/2305.09533v3
• Date: Sun, 13 Aug 2023 15:31:58 GMT
• Title: NightHazeFormer: Single Nighttime Haze Removal Using Prior Query Transformer
• Authors: Yun Liu, Zhongsheng Yan, Sixiang Chen, Tian Ye, Wenqi Ren and Erkang Chen
• Abstract summary: We propose an end-to-end transformer-based framework for nighttime haze removal, called NightHazeFormer. Our proposed approach consists of two stages: supervised pre-training and semi-supervised fine-tuning. Experiments on several synthetic and real-world datasets demonstrate the superiority of our NightHazeFormer over state-of-the-art nighttime haze removal methods.
• Score: 39.90066556289063
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Nighttime image dehazing is a challenging task due to the presence of multiple types of adverse degrading effects including glow, haze, blurry, noise, color distortion, and so on. However, most previous studies mainly focus on daytime image dehazing or partial degradations presented in nighttime hazy scenes, which may lead to unsatisfactory restoration results. In this paper, we propose an end-to-end transformer-based framework for nighttime haze removal, called NightHazeFormer. Our proposed approach consists of two stages: supervised pre-training and semi-supervised fine-tuning. During the pre-training stage, we introduce two powerful priors into the transformer decoder to generate the non-learnable prior queries, which guide the model to extract specific degradations. For the fine-tuning, we combine the generated pseudo ground truths with input real-world nighttime hazy images as paired images and feed into the synthetic domain to fine-tune the pre-trained model. This semi-supervised fine-tuning paradigm helps improve the generalization to real domain. In addition, we also propose a large-scale synthetic dataset called UNREAL-NH, to simulate the real-world nighttime haze scenarios comprehensively. Extensive experiments on several synthetic and real-world datasets demonstrate the superiority of our NightHazeFormer over state-of-the-art nighttime haze removal methods in terms of both visually and quantitatively.

Related papers

• Exploring Reliable Matching with Phase Enhancement for Night-time Semantic Segmentation [58.180226179087086]
  We propose a novel end-to-end optimized approach, named NightFormer, tailored for night-time semantic segmentation. Specifically, we design a pixel-level texture enhancement module to acquire texture-aware features hierarchically with phase enhancement and amplified attention. Our proposed method performs favorably against state-of-the-art night-time semantic segmentation methods.
  arXiv  Detail & Related papers  (2024-08-25T13:59:31Z)
• Self-Supervised Monocular Depth Estimation in the Dark: Towards Data Distribution Compensation [24.382795861986803]
  Using night images for self-supervision is unreliable because the photometric consistency assumption is usually violated in the videos taken under complex lighting conditions. We propose a self-supervised nighttime monocular depth estimation method that does not use any night images during training.
  arXiv  Detail & Related papers  (2024-04-22T03:39:03Z)
• A Semi-supervised Nighttime Dehazing Baseline with Spatial-Frequency Aware and Realistic Brightness Constraint [19.723367790947684]
  We propose a semi-supervised model for real-world nighttime dehazing. First, the spatial attention and frequency spectrum filtering are implemented as a spatial-frequency domain information interaction module. Second, a pseudo-label-based retraining strategy and a local window-based brightness loss for semi-supervised training process is designed to suppress haze and glow.
  arXiv  Detail & Related papers  (2024-03-27T13:27:02Z)
• NightHaze: Nighttime Image Dehazing via Self-Prior Learning [30.395213789178275]
  Masked autoencoder (MAE) shows that severe augmentation during training produces robust representations for high-level tasks. We propose a novel nighttime image dehazing method with self-prior learning. Our NightHaze, especially our MAE-like self-prior learning, shows that models trained with severe augmentation effectively improve the visibility of input haze images.
  arXiv  Detail & Related papers  (2024-03-12T08:35:42Z)
• ScatterNeRF: Seeing Through Fog with Physically-Based Inverse Neural Rendering [83.75284107397003]
  We introduce ScatterNeRF, a neural rendering method which renders scenes and decomposes the fog-free background. We propose a disentangled representation for the scattering volume and the scene objects, and learn the scene reconstruction with physics-inspired losses. We validate our method by capturing multi-view In-the-Wild data and controlled captures in a large-scale fog chamber.
  arXiv  Detail & Related papers  (2023-05-03T13:24:06Z)
• When the Sun Goes Down: Repairing Photometric Losses for All-Day Depth Estimation [47.617222712429026]
  We show how to use a combination of three techniques to allow the existing photometric losses to work for both day and nighttime images. First, we introduce a per-pixel neural intensity transformation to compensate for the light changes that occur between successive frames. Second, we predict a per-pixel residual flow map that we use to correct the reprojection correspondences induced by the estimated ego-motion and depth.
  arXiv  Detail & Related papers  (2022-06-28T09:29:55Z)
• Non-Homogeneous Haze Removal via Artificial Scene Prior and Bidimensional Graph Reasoning [52.07698484363237]
  We propose a Non-Homogeneous Haze Removal Network (NHRN) via artificial scene prior and bidimensional graph reasoning. Our method achieves superior performance over many state-of-the-art algorithms for both the single image dehazing and hazy image understanding tasks.
  arXiv  Detail & Related papers  (2021-04-05T13:04:44Z)
• Nighttime Dehazing with a Synthetic Benchmark [147.21955799938115]
  We propose a novel synthetic method called 3R to simulate nighttime hazy images from daytime clear images. We generate realistic nighttime hazy images by sampling real-world light colors from a prior empirical distribution. Experiment results demonstrate their superiority over state-of-the-art methods in terms of both image quality and runtime.
  arXiv  Detail & Related papers  (2020-08-10T02:16:46Z)
• Night-time Scene Parsing with a Large Real Dataset [67.11211537439152]
  We aim to address the night-time scene parsing (NTSP) problem, which has two main challenges. To tackle the scarcity of night-time data, we collect a novel labeled dataset, named it NightCity, of 4,297 real night-time images. We also propose an exposure-aware framework to address the NTSP problem through augmenting the segmentation process with explicitly learned exposure features.
  arXiv  Detail & Related papers  (2020-03-15T18:11:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.