Towards Dynamic and Small Objects Refinement for Unsupervised Domain Adaptative Nighttime Semantic Segmentation

• URL: http://arxiv.org/abs/2310.04747v2
• Date: Thu, 14 Mar 2024 08:59:58 GMT
• Title: Towards Dynamic and Small Objects Refinement for Unsupervised Domain Adaptative Nighttime Semantic Segmentation
• Authors: Jingyi Pan, Sihang Li, Yucheng Chen, Jinjing Zhu, Lin Wang,
• Abstract summary: Nighttime semantic segmentation plays a crucial role in practical applications, such as autonomous driving. UDA has shown the potential to address the challenges and achieved remarkable results for nighttime semantic segmentation. This paper proposes a novel UDA method that refines both label and feature levels for dynamic and small objects for nighttime semantic segmentation.
• Score: 10.770319457743192
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Nighttime semantic segmentation plays a crucial role in practical applications, such as autonomous driving, where it frequently encounters difficulties caused by inadequate illumination conditions and the absence of well-annotated datasets. Moreover, semantic segmentation models trained on daytime datasets often face difficulties in generalizing effectively to nighttime conditions. Unsupervised domain adaptation (UDA) has shown the potential to address the challenges and achieved remarkable results for nighttime semantic segmentation. However, existing methods still face limitations in 1) their reliance on style transfer or relighting models, which struggle to generalize to complex nighttime environments, and 2) their ignorance of dynamic and small objects like vehicles and poles, which are difficult to be directly learned from other domains. This paper proposes a novel UDA method that refines both label and feature levels for dynamic and small objects for nighttime semantic segmentation. First, we propose a dynamic and small object refinement module to complement the knowledge of dynamic and small objects from the source domain to target the nighttime domain. These dynamic and small objects are normally context-inconsistent in under-exposed conditions. Then, we design a feature prototype alignment module to reduce the domain gap by deploying contrastive learning between features and prototypes of the same class from different domains, while re-weighting the categories of dynamic and small objects. Extensive experiments on three benchmark datasets demonstrate that our method outperforms prior arts by a large margin for nighttime segmentation. Project page: https://rorisis.github.io/DSRNSS/.

Related papers

• Exploring Reliable Matching with Phase Enhancement for Night-time Semantic Segmentation [58.180226179087086]
  We propose a novel end-to-end optimized approach, named NightFormer, tailored for night-time semantic segmentation. Specifically, we design a pixel-level texture enhancement module to acquire texture-aware features hierarchically with phase enhancement and amplified attention. Our proposed method performs favorably against state-of-the-art night-time semantic segmentation methods.
  arXiv  Detail & Related papers  (2024-08-25T13:59:31Z)
• Prompt-Driven Dynamic Object-Centric Learning for Single Domain Generalization [61.64304227831361]
  Single-domain generalization aims to learn a model from single source domain data to achieve generalized performance on other unseen target domains. We propose a dynamic object-centric perception network based on prompt learning, aiming to adapt to the variations in image complexity.
  arXiv  Detail & Related papers  (2024-02-28T16:16:51Z)
• A Threefold Review on Deep Semantic Segmentation: Efficiency-oriented, Temporal and Depth-aware design [77.34726150561087]
  We conduct a survey on the most relevant and recent advances in Deep Semantic in the context of vision for autonomous vehicles. Our main objective is to provide a comprehensive discussion on the main methods, advantages, limitations, results and challenges faced from each perspective.
  arXiv  Detail & Related papers  (2023-03-08T01:29:55Z)
• ASAP: Accurate semantic segmentation for real time performance [3.5327983932835165]
  We propose an efficient feature fusion method, Feature Fusion with Different Norms (FFDN) FFDN utilizes rich global context of multi-level scale and vertical pooling module before self-attention. We achieve the mean Interaction of-union(mIoU) of 73.1 and the Frame Per Second(FPS) of 191, which are comparable results with state-of-the-arts on Cityscapes test datasets.
  arXiv  Detail & Related papers  (2022-10-04T02:35:53Z)
• Discovering Objects that Can Move [55.743225595012966]
  We study the problem of object discovery -- separating objects from the background without manual labels. Existing approaches utilize appearance cues, such as color, texture, and location, to group pixels into object-like regions. We choose to focus on dynamic objects -- entities that can move independently in the world.
  arXiv  Detail & Related papers  (2022-03-18T21:13:56Z)
• Reimagine BiSeNet for Real-Time Domain Adaptation in Semantic Segmentation [17.761939190746812]
  We look at the challenge of real-time semantic segmentation across domains. We train a model to act appropriately on real-world data even though it was trained on a synthetic realm. We employ a new lightweight and shallow discriminator that was specifically created for this purpose.
  arXiv  Detail & Related papers  (2021-10-22T08:39:28Z)
• Event-based Motion Segmentation with Spatio-Temporal Graph Cuts [51.17064599766138]
  We have developed a method to identify independently objects acquired with an event-based camera. The method performs on par or better than the state of the art without having to predetermine the number of expected moving objects.
  arXiv  Detail & Related papers  (2020-12-16T04:06:02Z)
• Unsupervised Domain Adaptation for Spatio-Temporal Action Localization [69.12982544509427]
  S-temporal action localization is an important problem in computer vision. We propose an end-to-end unsupervised domain adaptation algorithm. We show that significant performance gain can be achieved when spatial and temporal features are adapted separately or jointly.
  arXiv  Detail & Related papers  (2020-10-19T04:25:10Z)
• Lookahead Adversarial Learning for Near Real-Time Semantic Segmentation [2.538209532048867]
  We build a conditional adversarial network with a state-of-the-art segmentation model (DeepLabv3+) at its core. We focus on semantic segmentation models that run fast at inference for near real-time field applications.
  arXiv  Detail & Related papers  (2020-06-19T17:04:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.