Cross-Domain Correlation Distillation for Unsupervised Domain Adaptation
in Nighttime Semantic Segmentation
- URL: http://arxiv.org/abs/2205.00858v1
- Date: Mon, 2 May 2022 12:42:04 GMT
- Title: Cross-Domain Correlation Distillation for Unsupervised Domain Adaptation
in Nighttime Semantic Segmentation
- Authors: Huan Gao, Jichang Guo, Guoli Wang, Qian Zhang
- Abstract summary: We propose a novel domain adaptation framework via cross-domain correlation distillation, called CCDistill.
We extract the content and style knowledge contained in features, calculate the degree of inherent or illumination difference between two images.
Experiments on Dark Zurich and ACDC demonstrate that CCDistill achieves the state-of-the-art performance for nighttime semantic segmentation.
- Score: 17.874336775904272
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The performance of nighttime semantic segmentation is restricted by the poor
illumination and a lack of pixel-wise annotation, which severely limit its
application in autonomous driving. Existing works, e.g., using the twilight as
the intermediate target domain to perform the adaptation from daytime to
nighttime, may fail to cope with the inherent difference between datasets
caused by the camera equipment and the urban style. Faced with these two types
of domain shifts, i.e., the illumination and the inherent difference of the
datasets, we propose a novel domain adaptation framework via cross-domain
correlation distillation, called CCDistill. The invariance of illumination or
inherent difference between two images is fully explored so as to make up for
the lack of labels for nighttime images. Specifically, we extract the content
and style knowledge contained in features, calculate the degree of inherent or
illumination difference between two images. The domain adaptation is achieved
using the invariance of the same kind of difference. Extensive experiments on
Dark Zurich and ACDC demonstrate that CCDistill achieves the state-of-the-art
performance for nighttime semantic segmentation. Notably, our method is a
one-stage domain adaptation network which can avoid affecting the inference
time. Our implementation is available at https://github.com/ghuan99/CCDistill.
Related papers
- Exploring Reliable Matching with Phase Enhancement for Night-time Semantic Segmentation [58.180226179087086]
We propose a novel end-to-end optimized approach, named NightFormer, tailored for night-time semantic segmentation.
Specifically, we design a pixel-level texture enhancement module to acquire texture-aware features hierarchically with phase enhancement and amplified attention.
Our proposed method performs favorably against state-of-the-art night-time semantic segmentation methods.
arXiv Detail & Related papers (2024-08-25T13:59:31Z) - Distractors-Immune Representation Learning with Cross-modal Contrastive Regularization for Change Captioning [71.14084801851381]
Change captioning aims to succinctly describe the semantic change between a pair of similar images.
Most existing methods directly capture the difference between them, which risk obtaining error-prone difference features.
We propose a distractors-immune representation learning network that correlates the corresponding channels of two image representations.
arXiv Detail & Related papers (2024-07-16T13:00:33Z) - Similarity Min-Max: Zero-Shot Day-Night Domain Adaptation [52.923298434948606]
Low-light conditions not only hamper human visual experience but also degrade the model's performance on downstream vision tasks.
This paper challenges a more complicated scenario with border applicability, i.e., zero-shot day-night domain adaptation.
We propose a similarity min-max paradigm that considers them under a unified framework.
arXiv Detail & Related papers (2023-07-17T18:50:15Z) - Unsupervised Foggy Scene Understanding via Self Spatial-Temporal Label
Diffusion [51.11295961195151]
We exploit the characteristics of the foggy image sequence of driving scenes to densify the confident pseudo labels.
Based on the two discoveries of local spatial similarity and adjacent temporal correspondence of the sequential image data, we propose a novel Target-Domain driven pseudo label Diffusion scheme.
Our scheme helps the adaptive model achieve 51.92% and 53.84% mean intersection-over-union (mIoU) on two publicly available natural foggy datasets.
arXiv Detail & Related papers (2022-06-10T05:16:50Z) - Bi-Mix: Bidirectional Mixing for Domain Adaptive Nighttime Semantic
Segmentation [83.97914777313136]
In autonomous driving, learning a segmentation model that can adapt to various environmental conditions is crucial.
In this paper, we study the problem of Domain Adaptive Nighttime Semantic (DANSS), which aims to learn a discriminative nighttime model.
We propose a novel Bi-Mix framework for DANSS, which can contribute to both image translation and segmentation adaptation processes.
arXiv Detail & Related papers (2021-11-19T17:39:47Z) - Self-supervised Monocular Depth Estimation for All Day Images using
Domain Separation [17.066753214406525]
We propose a domain-separated network for self-supervised depth estimation of all-day images.
Our approach achieves state-of-the-art depth estimation results for all-day images on the challenging Oxford RobotCar dataset.
arXiv Detail & Related papers (2021-08-17T13:52:19Z) - DANNet: A One-Stage Domain Adaptation Network for Unsupervised Nighttime
Semantic Segmentation [18.43890050736093]
We propose a novel domain adaptation network (DANNet) for nighttime semantic segmentation.
It employs an adversarial training with a labeled daytime dataset and an unlabeled dataset that contains coarsely aligned day-night image pairs.
Our method achieves state-of-the-art performance for nighttime semantic segmentation.
arXiv Detail & Related papers (2021-04-22T02:49:28Z) - Map-Guided Curriculum Domain Adaptation and Uncertainty-Aware Evaluation
for Semantic Nighttime Image Segmentation [107.33492779588641]
We develop a curriculum framework to adapt semantic segmentation models from day to night without using nighttime annotations.
We also design a new evaluation framework to address the substantial uncertainty of semantics in nighttime images.
arXiv Detail & Related papers (2020-05-28T16:54:38Z) - HeatNet: Bridging the Day-Night Domain Gap in Semantic Segmentation with
Thermal Images [26.749261270690425]
Real-world driving scenarios entail adverse environmental conditions such as nighttime illumination or glare.
We propose a multimodal semantic segmentation model that can be applied during daytime and nighttime.
Besides RGB images, we leverage thermal images, making our network significantly more robust.
arXiv Detail & Related papers (2020-03-10T11:36:42Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.