Boosting Night-time Scene Parsing with Learnable Frequency
- URL: http://arxiv.org/abs/2208.14241v1
- Date: Tue, 30 Aug 2022 13:09:59 GMT
- Title: Boosting Night-time Scene Parsing with Learnable Frequency
- Authors: Zhifeng Xie, Sen Wang, Ke Xu, Zhizhong Zhang, Xin Tan, Yuan Xie,
Lizhuang Ma
- Abstract summary: Night-Time Scene Parsing (NTSP) is essential to many vision applications, especially for autonomous driving.
Most of the existing methods are proposed for day-time scene parsing.
We show that our method performs favorably against the state-of-the-art methods on the NightCity, NightCity+ and BDD100K-night datasets.
- Score: 53.05778451012621
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Night-Time Scene Parsing (NTSP) is essential to many vision applications,
especially for autonomous driving. Most of the existing methods are proposed
for day-time scene parsing. They rely on modeling pixel intensity-based spatial
contextual cues under even illumination. Hence, these methods do not perform
well in night-time scenes as such spatial contextual cues are buried in the
over-/under-exposed regions in night-time scenes. In this paper, we first
conduct an image frequency-based statistical experiment to interpret the
day-time and night-time scene discrepancies. We find that image frequency
distributions differ significantly between day-time and night-time scenes, and
understanding such frequency distributions is critical to NTSP problem. Based
on this, we propose to exploit the image frequency distributions for night-time
scene parsing. First, we propose a Learnable Frequency Encoder (LFE) to model
the relationship between different frequency coefficients to measure all
frequency components dynamically. Second, we propose a Spatial Frequency Fusion
module (SFF) that fuses both spatial and frequency information to guide the
extraction of spatial context features. Extensive experiments show that our
method performs favorably against the state-of-the-art methods on the
NightCity, NightCity+ and BDD100K-night datasets. In addition, we demonstrate
that our method can be applied to existing day-time scene parsing methods and
boost their performance on night-time scenes.
Related papers
- Exploring Reliable Matching with Phase Enhancement for Night-time Semantic Segmentation [58.180226179087086]
We propose a novel end-to-end optimized approach, named NightFormer, tailored for night-time semantic segmentation.
Specifically, we design a pixel-level texture enhancement module to acquire texture-aware features hierarchically with phase enhancement and amplified attention.
Our proposed method performs favorably against state-of-the-art night-time semantic segmentation methods.
arXiv Detail & Related papers (2024-08-25T13:59:31Z) - Sun Off, Lights On: Photorealistic Monocular Nighttime Simulation for Robust Semantic Perception [53.631644875171595]
Nighttime scenes are hard to semantically perceive with learned models and annotate for humans.
Our method, named Sun Off, Lights On (SOLO), is the first to perform nighttime simulation on single images in a photorealistic fashion by operating in 3D.
Not only is the visual quality and photorealism of our nighttime images superior to competing approaches including diffusion models, but the former images are also proven more beneficial for semantic nighttime segmentation in day-to-night adaptation.
arXiv Detail & Related papers (2024-07-29T18:00:09Z) - A Semi-supervised Nighttime Dehazing Baseline with Spatial-Frequency Aware and Realistic Brightness Constraint [19.723367790947684]
We propose a semi-supervised model for real-world nighttime dehazing.
First, the spatial attention and frequency spectrum filtering are implemented as a spatial-frequency domain information interaction module.
Second, a pseudo-label-based retraining strategy and a local window-based brightness loss for semi-supervised training process is designed to suppress haze and glow.
arXiv Detail & Related papers (2024-03-27T13:27:02Z) - Disentangled Contrastive Image Translation for Nighttime Surveillance [87.03178320662592]
Nighttime surveillance suffers from degradation due to poor illumination and arduous human annotations.
Existing methods rely on multi-spectral images to perceive objects in the dark, which are troubled by low resolution and color absence.
We argue that the ultimate solution for nighttime surveillance is night-to-day translation, or Night2Day.
This paper contributes a new surveillance dataset called NightSuR. It includes six scenes to support the study on nighttime surveillance.
arXiv Detail & Related papers (2023-07-11T06:40:27Z) - Spatial-Temporal Frequency Forgery Clue for Video Forgery Detection in
VIS and NIR Scenario [87.72258480670627]
Existing face forgery detection methods based on frequency domain find that the GAN forged images have obvious grid-like visual artifacts in the frequency spectrum compared to the real images.
This paper proposes a Cosine Transform-based Forgery Clue Augmentation Network (FCAN-DCT) to achieve a more comprehensive spatial-temporal feature representation.
arXiv Detail & Related papers (2022-07-05T09:27:53Z) - NightLab: A Dual-level Architecture with Hardness Detection for
Segmentation at Night [6.666707251631694]
We propose NightLab, a novel nighttime segmentation framework.
Models at two levels of granularity, i.e. image and regional, and each level is composed of light adaptation and segmentation modules.
Experiments on the NightCity and BDD100K datasets show NightLab achieves State-of-The-Art (SoTA) performance compared to concurrent methods.
arXiv Detail & Related papers (2022-04-12T05:50:22Z) - DANNet: A One-Stage Domain Adaptation Network for Unsupervised Nighttime
Semantic Segmentation [18.43890050736093]
We propose a novel domain adaptation network (DANNet) for nighttime semantic segmentation.
It employs an adversarial training with a labeled daytime dataset and an unlabeled dataset that contains coarsely aligned day-night image pairs.
Our method achieves state-of-the-art performance for nighttime semantic segmentation.
arXiv Detail & Related papers (2021-04-22T02:49:28Z) - Night-time Scene Parsing with a Large Real Dataset [67.11211537439152]
We aim to address the night-time scene parsing (NTSP) problem, which has two main challenges.
To tackle the scarcity of night-time data, we collect a novel labeled dataset, named it NightCity, of 4,297 real night-time images.
We also propose an exposure-aware framework to address the NTSP problem through augmenting the segmentation process with explicitly learned exposure features.
arXiv Detail & Related papers (2020-03-15T18:11:34Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.