Saliency Guided Image Warping for Unsupervised Domain Adaptation

• URL: http://arxiv.org/abs/2403.12712v2
• Date: Wed, 31 Jul 2024 02:33:31 GMT
• Title: Saliency Guided Image Warping for Unsupervised Domain Adaptation
• Authors: Shen Zheng, Anurag Ghosh, Srinivasa G. Narasimhan,
• Abstract summary: We improve UDA training by using in-place image warping to focus on salient object regions. We design instance-level saliency guidance to adaptively oversample object regions. Our approach improves adaptation across geographies, lighting, and weather conditions.
• Score: 19.144094571994756
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Driving is challenging in conditions like night, rain, and snow. The lack of good labeled datasets has hampered progress in scene understanding under such conditions. Unsupervised domain adaptation (UDA) using large labeled clear-day datasets is a promising research direction in such cases. Current UDA methods, however, treat all image pixels uniformly, leading to over-reliance on the dominant scene backgrounds (e.g., roads, sky, sidewalks) that appear dramatically different across domains. As a result, they struggle to learn effective features of smaller and often sparse foreground objects (e.g., people, vehicles, signs). In this work, we improve UDA training by using in-place image warping to focus on salient object regions. Our insight is that while backgrounds vary significantly across domains (e.g., snowy night vs. clear day), object appearances vary to a lesser extent. Therefore, we design instance-level saliency guidance to adaptively oversample object regions, which reduces adverse effects from background context and enhances backbone feature learning. We then unwarp the better learned features while adapting from source to target. Our approach improves adaptation across geographies, lighting, and weather conditions, and is agnostic to the task (segmentation, detection), domain adaptation algorithm, saliency guidance, and underlying model architecture. Result highlights include +6.1 mAP50 for BDD100K Clear $\rightarrow$ DENSE Foggy, +3.7 mAP50 for BDD100K Day $\rightarrow$ Night, +3.0 mAP50 for BDD100K Clear $\rightarrow$ Rainy, and +6.3 mIoU for Cityscapes $\rightarrow$ ACDC. Our method adds minimal training memory and incurs no additional inference latency. Please see Appendix for more results and analysis.

Related papers

• Pseudolabel guided pixels contrast for domain adaptive semantic segmentation [0.9831489366502301]
  Unsupervised domain adaptation (UDA) for semantic segmentation is a technique that uses virtual data with labels to train a model and adapts it to real data without labels. Some recent works use contrastive learning, which is a powerful method for self-supervised learning, to help with this technique. We propose a novel framework called Pseudo-label Guided Pixel Contrast (PGPC), which overcomes the disadvantages of previous methods.
  arXiv  Detail & Related papers  (2025-01-15T03:25:25Z)
• A-BDD: Leveraging Data Augmentations for Safe Autonomous Driving in Adverse Weather and Lighting [35.410769661423004]
  We present A-BDD, a large set of over 60,000 synthetically augmented images based on BDD100K. The dataset contains augmented data for rain, fog, overcast and sunglare/shadow with varying intensity levels. We provide evidence that data augmentations can play a pivotal role in closing performance gaps in adverse weather and lighting conditions.
  arXiv  Detail & Related papers  (2024-08-12T11:44:47Z)
• RoScenes: A Large-scale Multi-view 3D Dataset for Roadside Perception [98.76525636842177]
  RoScenes is the largest multi-view roadside perception dataset. Our dataset achieves surprising 21.13M 3D annotations within 64,000 $m2$.
  arXiv  Detail & Related papers  (2024-05-16T08:06:52Z)
• DTBS: Dual-Teacher Bi-directional Self-training for Domain Adaptation in Nighttime Semantic Segmentation [1.7205106391379026]
  Nighttime conditions pose a significant challenge for autonomous vehicle perception systems. Unsupervised domain adaptation (UDA) has been widely applied to semantic segmentation on such images. We introduce a one-stage Dual-Teacher Bi-directional Self-training (DTBS) framework for smooth knowledge transfer and feedback.
  arXiv  Detail & Related papers  (2024-01-02T06:56:57Z)
• T-UDA: Temporal Unsupervised Domain Adaptation in Sequential Point Clouds [2.5291108878852864]
  unsupervised domain adaptation (UDA) methods adapt models trained on one (source) domain with annotations available to another (target) domain for which only unannotated data are available. We introduce a novel domain adaptation method that leverages the best of both trends. Dubbed T-UDA for "temporal UDA", such a combination yields massive performance gains for the task of 3D semantic segmentation of driving scenes.
  arXiv  Detail & Related papers  (2023-09-15T10:47:12Z)
• MIC: Masked Image Consistency for Context-Enhanced Domain Adaptation [104.40114562948428]
  In unsupervised domain adaptation (UDA), a model trained on source data (e.g. synthetic) is adapted to target data (e.g. real-world) without access to target annotation. We propose a Masked Image Consistency (MIC) module to enhance UDA by learning spatial context relations of the target domain. MIC significantly improves the state-of-the-art performance across the different recognition tasks for synthetic-to-real, day-to-nighttime, and clear-to-adverse-weather UDA.
  arXiv  Detail & Related papers  (2022-12-02T17:29:32Z)
• Object Detection in Foggy Scenes by Embedding Depth and Reconstruction into Domain Adaptation [35.751428021375055]
  Most existing domain adaptation (DA) methods align the features based on the domain feature distributions. In our DA framework, we retain the depth and background information during the domain feature alignment. We propose an encoder-decoder framework to reconstruct the fog-free background image. Our method significantly outperforms the state-of-the-art method (47.6 mAP against the 44.3 mAP on the Foggy Cityscapes dataset)
  arXiv  Detail & Related papers  (2022-11-24T04:27:40Z)
• HRDA: Context-Aware High-Resolution Domain-Adaptive Semantic Segmentation [104.47737619026246]
  Unsupervised domain adaptation (UDA) aims to adapt a model trained on the source domain to the target domain. We propose HRDA, a multi-resolution training approach for UDA, that combines the strengths of small high-resolution crops to preserve fine segmentation details. It significantly improves the state-of-the-art performance by 5.5 mIoU for GTA-to-Cityscapes and 4.9 mIoU for Synthia-to-Cityscapes.
  arXiv  Detail & Related papers  (2022-04-27T18:00:26Z)
• Semi-Supervised Domain Adaptation with Prototypical Alignment and Consistency Learning [86.6929930921905]
  This paper studies how much it can help address domain shifts if we further have a few target samples labeled. To explore the full potential of landmarks, we incorporate a prototypical alignment (PA) module which calculates a target prototype for each class from the landmarks. Specifically, we severely perturb the labeled images, making PA non-trivial to achieve and thus promoting model generalizability.
  arXiv  Detail & Related papers  (2021-04-19T08:46:08Z)
• BoMuDANet: Unsupervised Adaptation for Visual Scene Understanding in Unstructured Driving Environments [54.22535063244038]
  We present an unsupervised adaptation approach for visual scene understanding in unstructured traffic environments. Our method is designed for unstructured real-world scenarios with dense and heterogeneous traffic consisting of cars, trucks, two-and three-wheelers, and pedestrians.
  arXiv  Detail & Related papers  (2020-09-22T08:25:44Z)
• Quasi-Dense Similarity Learning for Multiple Object Tracking [82.93471035675299]
  We present Quasi-Dense Similarity Learning, which densely samples hundreds of region proposals on a pair of images for contrastive learning. We can directly combine this similarity learning with existing detection methods to build Quasi-Dense Tracking (QDTrack)
  arXiv  Detail & Related papers  (2020-06-11T17:57:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.