DensePASS: Dense Panoramic Semantic Segmentation via Unsupervised Domain Adaptation with Attention-Augmented Context Exchange

• URL: http://arxiv.org/abs/2108.06383v1
• Date: Fri, 13 Aug 2021 20:15:46 GMT
• Title: DensePASS: Dense Panoramic Semantic Segmentation via Unsupervised Domain Adaptation with Attention-Augmented Context Exchange
• Authors: Chaoxiang Ma, Jiaming Zhang, Kailun Yang, Alina Roitberg and Rainer Stiefelhagen
• Abstract summary: We formalize the task of unsupervised domain adaptation for panoramic semantic segmentation. A network trained on labelled examples from the source domain of pinhole camera data is deployed in a different target domain of panoramic images. We build a generic framework for cross-domain panoramic semantic segmentation based on different variants of attention-augmented domain adaptation modules.
• Score: 32.29797061415896
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Intelligent vehicles clearly benefit from the expanded Field of View (FoV) of the 360-degree sensors, but the vast majority of available semantic segmentation training images are captured with pinhole cameras. In this work, we look at this problem through the lens of domain adaptation and bring panoramic semantic segmentation to a setting, where labelled training data originates from a different distribution of conventional pinhole camera images. First, we formalize the task of unsupervised domain adaptation for panoramic semantic segmentation, where a network trained on labelled examples from the source domain of pinhole camera data is deployed in a different target domain of panoramic images, for which no labels are available. To validate this idea, we collect and publicly release DensePASS - a novel densely annotated dataset for panoramic segmentation under cross-domain conditions, specifically built to study the Pinhole-to-Panoramic transfer and accompanied with pinhole camera training examples obtained from Cityscapes. DensePASS covers both, labelled- and unlabelled 360-degree images, with the labelled data comprising 19 classes which explicitly fit the categories available in the source domain (i.e. pinhole) data. To meet the challenge of domain shift, we leverage the current progress of attention-based mechanisms and build a generic framework for cross-domain panoramic semantic segmentation based on different variants of attention-augmented domain adaptation modules. Our framework facilitates information exchange at local- and global levels when learning the domain correspondences and improves the domain adaptation performance of two standard segmentation networks by 6.05% and 11.26% in Mean IoU.

Related papers

• Multi-source Domain Adaptation for Panoramic Semantic Segmentation [22.367890439050786]
  We propose a new task of multi-source domain adaptation for panoramic semantic segmentation. We aim to utilize both real pinhole synthetic panoramic images in the source domains, enabling the segmentation model to perform well on unlabeled real panoramic images. DTA4PASS converts all pinhole images in the source domains into panoramic-like images, and then aligns the converted source domains with the target domain.
  arXiv  Detail & Related papers  (2024-08-29T12:00:11Z)
• I2F: A Unified Image-to-Feature Approach for Domain Adaptive Semantic Segmentation [55.633859439375044]
  Unsupervised domain adaptation (UDA) for semantic segmentation is a promising task freeing people from heavy annotation work. Key idea to tackle this problem is to perform both image-level and feature-level adaptation jointly. This paper proposes a novel UDA pipeline for semantic segmentation that unifies image-level and feature-level adaptation.
  arXiv  Detail & Related papers  (2023-01-03T15:19:48Z)
• PiPa: Pixel- and Patch-wise Self-supervised Learning for Domain Adaptative Semantic Segmentation [100.6343963798169]
  Unsupervised Domain Adaptation (UDA) aims to enhance the generalization of the learned model to other domains. We propose a unified pixel- and patch-wise self-supervised learning framework, called PiPa, for domain adaptive semantic segmentation.
  arXiv  Detail & Related papers  (2022-11-14T18:31:24Z)
• Multi-Scale Multi-Target Domain Adaptation for Angle Closure Classification [50.658613573816254]
  We propose a novel Multi-scale Multi-target Domain Adversarial Network (M2DAN) for angle closure classification. Based on these domain-invariant features at different scales, the deep model trained on the source domain is able to classify angle closure on multiple target domains.
  arXiv  Detail & Related papers  (2022-08-25T15:27:55Z)
• Panoramic Panoptic Segmentation: Insights Into Surrounding Parsing for Mobile Agents via Unsupervised Contrastive Learning [93.6645991946674]
  We introduce panoramic panoptic segmentation, as the most holistic scene understanding. A complete surrounding understanding provides a maximum of information to a mobile agent. We propose a framework which allows model training on standard pinhole images and transfers the learned features to a different domain.
  arXiv  Detail & Related papers  (2022-06-21T20:07:15Z)
• HYLDA: End-to-end Hybrid Learning Domain Adaptation for LiDAR Semantic Segmentation [13.87939140266266]
  This paper addresses the problem of training a LiDAR semantic segmentation network using a fully-labeled source dataset and a target dataset that only has a small number of labels. We develop a novel image-to-image translation engine, and couple it with a LiDAR semantic segmentation network, resulting in an integrated domain adaptation architecture we call HYLDA.
  arXiv  Detail & Related papers  (2022-01-14T18:13:09Z)
• Transfer beyond the Field of View: Dense Panoramic Semantic Segmentation via Unsupervised Domain Adaptation [30.104947024614127]
  We formalize the task of unsupervised domain adaptation for panoramic semantic segmentation. DensePASS is a novel dataset for panoramic segmentation under cross-domain conditions. We introduce P2PDA - a generic framework for Pinhole-to-Panoramic semantic segmentation.
  arXiv  Detail & Related papers  (2021-10-21T11:22:05Z)
• DRANet: Disentangling Representation and Adaptation Networks for Unsupervised Cross-Domain Adaptation [23.588766224169493]
  DRANet is a network architecture that disentangles image representations and transfers the visual attributes in a latent space for unsupervised cross-domain adaptation. Our model encodes individual representations of content (scene structure) and style (artistic appearance) from both source and target images. It adapts the domain by incorporating the transferred style factor into the content factor along with learnable weights specified for each domain.
  arXiv  Detail & Related papers  (2021-03-24T18:54:23Z)
• Domain Adaptation on Semantic Segmentation for Aerial Images [3.946367634483361]
  We propose a novel unsupervised domain adaptation framework to address domain shift in semantic image segmentation. We also apply entropy minimization on the target domain to produce high-confident prediction. We show improvement over state-of-the-art methods in terms of various metrics.
  arXiv  Detail & Related papers  (2020-12-03T20:58:27Z)
• DoFE: Domain-oriented Feature Embedding for Generalizable Fundus Image Segmentation on Unseen Datasets [96.92018649136217]
  We present a novel Domain-oriented Feature Embedding (DoFE) framework to improve the generalization ability of CNNs on unseen target domains. Our DoFE framework dynamically enriches the image features with additional domain prior knowledge learned from multi-source domains. Our framework generates satisfying segmentation results on unseen datasets and surpasses other domain generalization and network regularization methods.
  arXiv  Detail & Related papers  (2020-10-13T07:28:39Z)
• Phase Consistent Ecological Domain Adaptation [76.75730500201536]
  We focus on the task of semantic segmentation, where annotated synthetic data are aplenty, but annotating real data is laborious. The first criterion, inspired by visual psychophysics, is that the map between the two image domains be phase-preserving. The second criterion aims to leverage ecological statistics, or regularities in the scene which are manifest in any image of it, regardless of the characteristics of the illuminant or the imaging sensor.
  arXiv  Detail & Related papers  (2020-04-10T06:58:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.