Plugging Self-Supervised Monocular Depth into Unsupervised Domain Adaptation for Semantic Segmentation

• URL: http://arxiv.org/abs/2110.06685v1
• Date: Wed, 13 Oct 2021 12:48:51 GMT
• Title: Plugging Self-Supervised Monocular Depth into Unsupervised Domain Adaptation for Semantic Segmentation
• Authors: Adriano Cardace, Luca De Luigi, Pierluigi Zama Ramirez, Samuele Salti, Luigi Di Stefano
• Abstract summary: We propose to exploit self-supervised monocular depth estimation to improve UDA for semantic segmentation. Our whole proposal allows for achieving state-of-the-art performance (58.8 mIoU) in the GTA5->CS benchmark benchmark.
• Score: 19.859764556851434
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Although recent semantic segmentation methods have made remarkable progress, they still rely on large amounts of annotated training data, which are often infeasible to collect in the autonomous driving scenario. Previous works usually tackle this issue with Unsupervised Domain Adaptation (UDA), which entails training a network on synthetic images and applying the model to real ones while minimizing the discrepancy between the two domains. Yet, these techniques do not consider additional information that may be obtained from other tasks. Differently, we propose to exploit self-supervised monocular depth estimation to improve UDA for semantic segmentation. On one hand, we deploy depth to realize a plug-in component which can inject complementary geometric cues into any existing UDA method. We further rely on depth to generate a large and varied set of samples to Self-Train the final model. Our whole proposal allows for achieving state-of-the-art performance (58.8 mIoU) in the GTA5->CS benchmark benchmark. Code is available at https://github.com/CVLAB-Unibo/d4-dbst.

Related papers

• MICDrop: Masking Image and Depth Features via Complementary Dropout for Domain-Adaptive Semantic Segmentation [155.0797148367653]
  Unsupervised Domain Adaptation (UDA) is the task of bridging the domain gap between a labeled source domain and an unlabeled target domain. We propose to leverage geometric information, i.e., depth predictions, as depth discontinuities often coincide with segmentation boundaries. We show that our method can be plugged into various recent UDA methods and consistently improve results across standard UDA benchmarks.
  arXiv  Detail & Related papers  (2024-08-29T12:15:10Z)
• T-UDA: Temporal Unsupervised Domain Adaptation in Sequential Point Clouds [2.5291108878852864]
  unsupervised domain adaptation (UDA) methods adapt models trained on one (source) domain with annotations available to another (target) domain for which only unannotated data are available. We introduce a novel domain adaptation method that leverages the best of both trends. Dubbed T-UDA for "temporal UDA", such a combination yields massive performance gains for the task of 3D semantic segmentation of driving scenes.
  arXiv  Detail & Related papers  (2023-09-15T10:47:12Z)
• ALSO: Automotive Lidar Self-supervision by Occupancy estimation [70.70557577874155]
  We propose a new self-supervised method for pre-training the backbone of deep perception models operating on point clouds. The core idea is to train the model on a pretext task which is the reconstruction of the surface on which the 3D points are sampled. The intuition is that if the network is able to reconstruct the scene surface, given only sparse input points, then it probably also captures some fragments of semantic information.
  arXiv  Detail & Related papers  (2022-12-12T13:10:19Z)
• Self-Distillation for Unsupervised 3D Domain Adaptation [15.660045080090711]
  We propose an Unsupervised Domain Adaptation (UDA) method for point cloud classification in 3D vision. In this work, we focus on obtaining a discriminative feature space for the target domain enforcing consistency between a point cloud and its augmented version. We propose a novel iterative self-training methodology that exploits Graph Neural Networks in the UDA context to refine pseudo-labels.
  arXiv  Detail & Related papers  (2022-10-15T08:37:02Z)
• Threshold-adaptive Unsupervised Focal Loss for Domain Adaptation of Semantic Segmentation [25.626882426111198]
  Unsupervised domain adaptation (UDA) for semantic segmentation has recently gained increasing research attention. In this paper, we propose a novel two-stage entropy-based UDA method for semantic segmentation. Our method achieves state-of-the-art 58.4% and 59.6% mIoUs on SYNTHIA-to-Cityscapes and GTA5-to-Cityscapes using DeepLabV2 and competitive performance using the lightweight BiSeNet.
  arXiv  Detail & Related papers  (2022-08-23T03:48:48Z)
• HRDA: Context-Aware High-Resolution Domain-Adaptive Semantic Segmentation [104.47737619026246]
  Unsupervised domain adaptation (UDA) aims to adapt a model trained on the source domain to the target domain. We propose HRDA, a multi-resolution training approach for UDA, that combines the strengths of small high-resolution crops to preserve fine segmentation details. It significantly improves the state-of-the-art performance by 5.5 mIoU for GTA-to-Cityscapes and 4.9 mIoU for Synthia-to-Cityscapes.
  arXiv  Detail & Related papers  (2022-04-27T18:00:26Z)
• Improving Semi-Supervised and Domain-Adaptive Semantic Segmentation with Self-Supervised Depth Estimation [94.16816278191477]
  We present a framework for semi-adaptive and domain-supervised semantic segmentation. It is enhanced by self-supervised monocular depth estimation trained only on unlabeled image sequences. We validate the proposed model on the Cityscapes dataset.
  arXiv  Detail & Related papers  (2021-08-28T01:33:38Z)
• Domain Adaptive Semantic Segmentation with Self-Supervised Depth Estimation [84.34227665232281]
  Domain adaptation for semantic segmentation aims to improve the model performance in the presence of a distribution shift between source and target domain. We leverage the guidance from self-supervised depth estimation, which is available on both domains, to bridge the domain gap. We demonstrate the effectiveness of our proposed approach on the benchmark tasks SYNTHIA-to-Cityscapes and GTA-to-Cityscapes.
  arXiv  Detail & Related papers  (2021-04-28T07:47:36Z)
• Semi-Supervised Domain Adaptation with Prototypical Alignment and Consistency Learning [86.6929930921905]
  This paper studies how much it can help address domain shifts if we further have a few target samples labeled. To explore the full potential of landmarks, we incorporate a prototypical alignment (PA) module which calculates a target prototype for each class from the landmarks. Specifically, we severely perturb the labeled images, making PA non-trivial to achieve and thus promoting model generalizability.
  arXiv  Detail & Related papers  (2021-04-19T08:46:08Z)
• Geometric Unsupervised Domain Adaptation for Semantic Segmentation [35.492127636785824]
  A simulator can efficiently generate large amounts of labeled synthetic data with perfect supervision for hard-to-label tasks like semantic segmentation. We propose to use self-supervised monocular depth estimation as a proxy task to bridge this gap and improve sim-to-real unsupervised domain adaptation. Our method scales well with the quality and quantity of synthetic data while also improving depth prediction.
  arXiv  Detail & Related papers  (2021-03-30T21:33:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.