Map-Guided Curriculum Domain Adaptation and Uncertainty-Aware Evaluation
for Semantic Nighttime Image Segmentation
- URL: http://arxiv.org/abs/2005.14553v2
- Date: Thu, 7 Jan 2021 15:26:43 GMT
- Title: Map-Guided Curriculum Domain Adaptation and Uncertainty-Aware Evaluation
for Semantic Nighttime Image Segmentation
- Authors: Christos Sakaridis, Dengxin Dai, Luc Van Gool
- Abstract summary: We develop a curriculum framework to adapt semantic segmentation models from day to night without using nighttime annotations.
We also design a new evaluation framework to address the substantial uncertainty of semantics in nighttime images.
- Score: 107.33492779588641
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We address the problem of semantic nighttime image segmentation and improve
the state-of-the-art, by adapting daytime models to nighttime without using
nighttime annotations. Moreover, we design a new evaluation framework to
address the substantial uncertainty of semantics in nighttime images. Our
central contributions are: 1) a curriculum framework to gradually adapt
semantic segmentation models from day to night through progressively darker
times of day, exploiting cross-time-of-day correspondences between daytime
images from a reference map and dark images to guide the label inference in the
dark domains; 2) a novel uncertainty-aware annotation and evaluation framework
and metric for semantic segmentation, including image regions beyond human
recognition capability in the evaluation in a principled fashion; 3) the Dark
Zurich dataset, comprising 2416 unlabeled nighttime and 2920 unlabeled twilight
images with correspondences to their daytime counterparts plus a set of 201
nighttime images with fine pixel-level annotations created with our protocol,
which serves as a first benchmark for our novel evaluation. Experiments show
that our map-guided curriculum adaptation significantly outperforms
state-of-the-art methods on nighttime sets both for standard metrics and our
uncertainty-aware metric. Furthermore, our uncertainty-aware evaluation reveals
that selective invalidation of predictions can improve results on data with
ambiguous content such as our benchmark and profit safety-oriented applications
involving invalid inputs.
Related papers
- Self-Supervised Monocular Depth Estimation in the Dark: Towards Data Distribution Compensation [24.382795861986803]
Using night images for self-supervision is unreliable because the photometric consistency assumption is usually violated in the videos taken under complex lighting conditions.
We propose a self-supervised nighttime monocular depth estimation method that does not use any night images during training.
arXiv Detail & Related papers (2024-04-22T03:39:03Z) - Introspective Deep Metric Learning [91.47907685364036]
We propose an introspective deep metric learning framework for uncertainty-aware comparisons of images.
The proposed IDML framework improves the performance of deep metric learning through uncertainty modeling.
arXiv Detail & Related papers (2023-09-11T16:21:13Z) - Similarity Min-Max: Zero-Shot Day-Night Domain Adaptation [52.923298434948606]
Low-light conditions not only hamper human visual experience but also degrade the model's performance on downstream vision tasks.
This paper challenges a more complicated scenario with border applicability, i.e., zero-shot day-night domain adaptation.
We propose a similarity min-max paradigm that considers them under a unified framework.
arXiv Detail & Related papers (2023-07-17T18:50:15Z) - From colouring-in to pointillism: revisiting semantic segmentation
supervision [48.637031591058175]
We propose a pointillist approach for semantic segmentation annotation, where only point-wise yes/no questions are answered.
We collected and released 22.6M point labels over 4,171 classes on the Open Images dataset.
arXiv Detail & Related papers (2022-10-25T16:42:03Z) - Introspective Deep Metric Learning for Image Retrieval [80.29866561553483]
We argue that a good similarity model should consider the semantic discrepancies with caution to better deal with ambiguous images for more robust training.
We propose to represent an image using not only a semantic embedding but also an accompanying uncertainty embedding, which describes the semantic characteristics and ambiguity of an image, respectively.
The proposed IDML framework improves the performance of deep metric learning through uncertainty modeling and attains state-of-the-art results on the widely used CUB-200-2011, Cars196, and Stanford Online Products datasets.
arXiv Detail & Related papers (2022-05-09T17:51:44Z) - Cross-Domain Correlation Distillation for Unsupervised Domain Adaptation
in Nighttime Semantic Segmentation [17.874336775904272]
We propose a novel domain adaptation framework via cross-domain correlation distillation, called CCDistill.
We extract the content and style knowledge contained in features, calculate the degree of inherent or illumination difference between two images.
Experiments on Dark Zurich and ACDC demonstrate that CCDistill achieves the state-of-the-art performance for nighttime semantic segmentation.
arXiv Detail & Related papers (2022-05-02T12:42:04Z) - Self-supervised Monocular Depth Estimation for All Day Images using
Domain Separation [17.066753214406525]
We propose a domain-separated network for self-supervised depth estimation of all-day images.
Our approach achieves state-of-the-art depth estimation results for all-day images on the challenging Oxford RobotCar dataset.
arXiv Detail & Related papers (2021-08-17T13:52:19Z) - DANNet: A One-Stage Domain Adaptation Network for Unsupervised Nighttime
Semantic Segmentation [18.43890050736093]
We propose a novel domain adaptation network (DANNet) for nighttime semantic segmentation.
It employs an adversarial training with a labeled daytime dataset and an unlabeled dataset that contains coarsely aligned day-night image pairs.
Our method achieves state-of-the-art performance for nighttime semantic segmentation.
arXiv Detail & Related papers (2021-04-22T02:49:28Z) - Unsupervised Monocular Depth Estimation for Night-time Images using
Adversarial Domain Feature Adaptation [17.067988025947024]
We look into the problem of estimating per-pixel depth maps from unconstrained RGB monocular night-time images.
The state-of-the-art day-time depth estimation methods fail miserably when tested with night-time images.
We propose to solve this problem by posing it as a domain adaptation problem where a network trained with day-time images is adapted to work for night-time images.
arXiv Detail & Related papers (2020-10-03T17:55:16Z) - Night-time Scene Parsing with a Large Real Dataset [67.11211537439152]
We aim to address the night-time scene parsing (NTSP) problem, which has two main challenges.
To tackle the scarcity of night-time data, we collect a novel labeled dataset, named it NightCity, of 4,297 real night-time images.
We also propose an exposure-aware framework to address the NTSP problem through augmenting the segmentation process with explicitly learned exposure features.
arXiv Detail & Related papers (2020-03-15T18:11:34Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.