GPS-GLASS: Learning Nighttime Semantic Segmentation Using Daytime Video and GPS data

• URL: http://arxiv.org/abs/2207.13297v5
• Date: Fri, 18 Aug 2023 06:38:32 GMT
• Title: GPS-GLASS: Learning Nighttime Semantic Segmentation Using Daytime Video and GPS data
• Authors: Hongjae Lee, Changwoo Han, Jun-Sang Yoo, Seung-Won Jung
• Abstract summary: Nighttime semantic segmentation is especially challenging due to a lack of annotated nighttime images. We propose a novel GPS-based training framework for nighttime semantic segmentation. Experimental results demonstrate the effectiveness of the proposed method on several nighttime semantic segmentation datasets.
• Score: 15.430918080412518
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Semantic segmentation for autonomous driving should be robust against various in-the-wild environments. Nighttime semantic segmentation is especially challenging due to a lack of annotated nighttime images and a large domain gap from daytime images with sufficient annotation. In this paper, we propose a novel GPS-based training framework for nighttime semantic segmentation. Given GPS-aligned pairs of daytime and nighttime images, we perform cross-domain correspondence matching to obtain pixel-level pseudo supervision. Moreover, we conduct flow estimation between daytime video frames and apply GPS-based scaling to acquire another pixel-level pseudo supervision. Using these pseudo supervisions with a confidence map, we train a nighttime semantic segmentation network without any annotation from nighttime images. Experimental results demonstrate the effectiveness of the proposed method on several nighttime semantic segmentation datasets. Our source code is available at https://github.com/jimmy9704/GPS-GLASS.

Related papers

• Exploring Reliable Matching with Phase Enhancement for Night-time Semantic Segmentation [58.180226179087086]
  We propose a novel end-to-end optimized approach, named NightFormer, tailored for night-time semantic segmentation. Specifically, we design a pixel-level texture enhancement module to acquire texture-aware features hierarchically with phase enhancement and amplified attention. Our proposed method performs favorably against state-of-the-art night-time semantic segmentation methods.
  arXiv  Detail & Related papers  (2024-08-25T13:59:31Z)
• Learning Semantic Segmentation with Query Points Supervision on Aerial Images [57.09251327650334]
  We present a weakly supervised learning algorithm to train semantic segmentation algorithms. Our proposed approach performs accurate semantic segmentation and improves efficiency by significantly reducing the cost and time required for manual annotation.
  arXiv  Detail & Related papers  (2023-09-11T14:32:04Z)
• CMDA: Cross-Modality Domain Adaptation for Nighttime Semantic Segmentation [21.689985575213512]
  We propose a novel unsupervised Cross-Modality Domain Adaptation (CMDA) framework to leverage multi-modality (Images and Events) information for nighttime semantic segmentation. In CMDA, we design the Image Motion-Extractor to extract motion information and the Image Content-Extractor to extract content information from images. We introduce the first image-event nighttime semantic segmentation dataset.
  arXiv  Detail & Related papers  (2023-07-29T09:29:09Z)
• Disentangled Contrastive Image Translation for Nighttime Surveillance [87.03178320662592]
  Nighttime surveillance suffers from degradation due to poor illumination and arduous human annotations. Existing methods rely on multi-spectral images to perceive objects in the dark, which are troubled by low resolution and color absence. We argue that the ultimate solution for nighttime surveillance is night-to-day translation, or Night2Day. This paper contributes a new surveillance dataset called NightSuR. It includes six scenes to support the study on nighttime surveillance.
  arXiv  Detail & Related papers  (2023-07-11T06:40:27Z)
• LoopDA: Constructing Self-loops to Adapt Nighttime Semantic Segmentation [5.961294477200831]
  We propose LoopDA for domain adaptive nighttime semantic segmentation. Our model outperforms prior methods on Dark Zurich and Nighttime Driving datasets for semantic segmentation.
  arXiv  Detail & Related papers  (2022-11-21T21:46:05Z)
• Boosting Night-time Scene Parsing with Learnable Frequency [53.05778451012621]
  Night-Time Scene Parsing (NTSP) is essential to many vision applications, especially for autonomous driving. Most of the existing methods are proposed for day-time scene parsing. We show that our method performs favorably against the state-of-the-art methods on the NightCity, NightCity+ and BDD100K-night datasets.
  arXiv  Detail & Related papers  (2022-08-30T13:09:59Z)
• Improving Nighttime Driving-Scene Segmentation via Dual Image-adaptive Learnable Filters [27.960081476653023]
  We present an add-on module called dual image-adaptive learnable filters (DIAL-Filters) to improve the semantic segmentation in nighttime driving conditions. DIAL-Filters consist of two parts, including an image-adaptive processing module (IAPM) and a learnable guided filter (LGF)
  arXiv  Detail & Related papers  (2022-07-04T11:24:10Z)
• NightLab: A Dual-level Architecture with Hardness Detection for Segmentation at Night [6.666707251631694]
  We propose NightLab, a novel nighttime segmentation framework. Models at two levels of granularity, i.e. image and regional, and each level is composed of light adaptation and segmentation modules. Experiments on the NightCity and BDD100K datasets show NightLab achieves State-of-The-Art (SoTA) performance compared to concurrent methods.
  arXiv  Detail & Related papers  (2022-04-12T05:50:22Z)
• Bi-Mix: Bidirectional Mixing for Domain Adaptive Nighttime Semantic Segmentation [83.97914777313136]
  In autonomous driving, learning a segmentation model that can adapt to various environmental conditions is crucial. In this paper, we study the problem of Domain Adaptive Nighttime Semantic (DANSS), which aims to learn a discriminative nighttime model. We propose a novel Bi-Mix framework for DANSS, which can contribute to both image translation and segmentation adaptation processes.
  arXiv  Detail & Related papers  (2021-11-19T17:39:47Z)
• DANNet: A One-Stage Domain Adaptation Network for Unsupervised Nighttime Semantic Segmentation [18.43890050736093]
  We propose a novel domain adaptation network (DANNet) for nighttime semantic segmentation. It employs an adversarial training with a labeled daytime dataset and an unlabeled dataset that contains coarsely aligned day-night image pairs. Our method achieves state-of-the-art performance for nighttime semantic segmentation.
  arXiv  Detail & Related papers  (2021-04-22T02:49:28Z)
• Map-Guided Curriculum Domain Adaptation and Uncertainty-Aware Evaluation for Semantic Nighttime Image Segmentation [107.33492779588641]
  We develop a curriculum framework to adapt semantic segmentation models from day to night without using nighttime annotations. We also design a new evaluation framework to address the substantial uncertainty of semantics in nighttime images.
  arXiv  Detail & Related papers  (2020-05-28T16:54:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.