EDAPS: Enhanced Domain-Adaptive Panoptic Segmentation
- URL: http://arxiv.org/abs/2304.14291v2
- Date: Thu, 21 Dec 2023 20:54:54 GMT
- Title: EDAPS: Enhanced Domain-Adaptive Panoptic Segmentation
- Authors: Suman Saha, Lukas Hoyer, Anton Obukhov, Dengxin Dai and Luc Van Gool
- Abstract summary: We study the panoptic network design and propose a novel architecture (EDAPS) designed explicitly for domain-adaptive panoptic segmentation.
EDAPS significantly improves the state-of-the-art performance for panoptic segmentation UDA by a large margin of 20% on SYNTHIA-to-Cityscapes and even 72% on the more challenging SYNTHIA-to-Mapillary Vistas.
- Score: 93.25977558780896
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: With autonomous industries on the rise, domain adaptation of the visual
perception stack is an important research direction due to the cost savings
promise. Much prior art was dedicated to domain-adaptive semantic segmentation
in the synthetic-to-real context. Despite being a crucial output of the
perception stack, panoptic segmentation has been largely overlooked by the
domain adaptation community. Therefore, we revisit well-performing domain
adaptation strategies from other fields, adapt them to panoptic segmentation,
and show that they can effectively enhance panoptic domain adaptation. Further,
we study the panoptic network design and propose a novel architecture (EDAPS)
designed explicitly for domain-adaptive panoptic segmentation. It uses a
shared, domain-robust transformer encoder to facilitate the joint adaptation of
semantic and instance features, but task-specific decoders tailored for the
specific requirements of both domain-adaptive semantic and instance
segmentation. As a result, the performance gap seen in challenging panoptic
benchmarks is substantially narrowed. EDAPS significantly improves the
state-of-the-art performance for panoptic segmentation UDA by a large margin of
20% on SYNTHIA-to-Cityscapes and even 72% on the more challenging
SYNTHIA-to-Mapillary Vistas. The implementation is available at
https://github.com/susaha/edaps.
Related papers
- Language-Guided Instance-Aware Domain-Adaptive Panoptic Segmentation [44.501770535446624]
Key challenge in panoptic domain adaptation is reducing the domain gap between a labeled source and an unlabeled target domain.
We focus on incorporating instance-level adaptation via a novel cross-domain mixing strategy IMix.
We present an end-to-end model incorporating these two mechanisms called LIDAPS, achieving state-of-the-art results on all popular panoptic UDA benchmarks.
arXiv Detail & Related papers (2024-04-04T20:42:49Z) - Exploring Consistency in Cross-Domain Transformer for Domain Adaptive
Semantic Segmentation [51.10389829070684]
Domain gap can cause discrepancies in self-attention.
Due to this gap, the transformer attends to spurious regions or pixels, which deteriorates accuracy on the target domain.
We propose adaptation on attention maps with cross-domain attention layers.
arXiv Detail & Related papers (2022-11-27T02:40:33Z) - UniDAformer: Unified Domain Adaptive Panoptic Segmentation Transformer
via Hierarchical Mask Calibration [49.16591283724376]
We design UniDAformer, a unified domain adaptive panoptic segmentation transformer that is simple but can achieve domain adaptive instance segmentation and semantic segmentation simultaneously within a single network.
UniDAformer introduces Hierarchical Mask (HMC) that rectifies inaccurate predictions at the level of regions, superpixels and annotated pixels via online self-training on the fly.
It has three unique features: 1) it enables unified domain adaptive panoptic adaptation; 2) it mitigates false predictions and improves domain adaptive panoptic segmentation effectively; 3) it is end-to-end trainable with a much simpler training and inference pipeline.
arXiv Detail & Related papers (2022-06-30T07:32:23Z) - Amplitude Spectrum Transformation for Open Compound Domain Adaptive
Semantic Segmentation [62.68759523116924]
Open compound domain adaptation (OCDA) has emerged as a practical adaptation setting.
We propose a novel feature space Amplitude Spectrum Transformation (AST)
arXiv Detail & Related papers (2022-02-09T05:40:34Z) - Cross-View Regularization for Domain Adaptive Panoptic Segmentation [32.77436219094282]
We design a domain adaptive panoptic segmentation network that exploits inter-style consistency and inter-task regularization.
Our proposed network achieves superior segmentation performance as compared with the state-of-the-art.
arXiv Detail & Related papers (2021-03-03T18:29:23Z) - Affinity Space Adaptation for Semantic Segmentation Across Domains [57.31113934195595]
In this paper, we address the problem of unsupervised domain adaptation (UDA) in semantic segmentation.
Motivated by the fact that source and target domain have invariant semantic structures, we propose to exploit such invariance across domains.
We develop two affinity space adaptation strategies: affinity space cleaning and adversarial affinity space alignment.
arXiv Detail & Related papers (2020-09-26T10:28:11Z) - Unsupervised Bidirectional Cross-Modality Adaptation via Deeply
Synergistic Image and Feature Alignment for Medical Image Segmentation [73.84166499988443]
We present a novel unsupervised domain adaptation framework, named as Synergistic Image and Feature Alignment (SIFA)
Our proposed SIFA conducts synergistic alignment of domains from both image and feature perspectives.
Experimental results on two different tasks demonstrate that our SIFA method is effective in improving segmentation performance on unlabeled target images.
arXiv Detail & Related papers (2020-02-06T13:49:47Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.