To Adapt or Not to Adapt? Real-Time Adaptation for Semantic Segmentation

• URL: http://arxiv.org/abs/2307.15063v2
• Date: Mon, 7 Aug 2023 10:43:33 GMT
• Title: To Adapt or Not to Adapt? Real-Time Adaptation for Semantic Segmentation
• Abstract summary: HAMLET is a Hardware-Aware Modular Least Expensive Training framework for real-time domain adaptation. Our approach is capable of performing semantic segmentation while simultaneously adapting at more than 29FPS on a single consumer-grade GPU.
• Score: 45.74645069886128
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The goal of Online Domain Adaptation for semantic segmentation is to handle unforeseeable domain changes that occur during deployment, like sudden weather events. However, the high computational costs associated with brute-force adaptation make this paradigm unfeasible for real-world applications. In this paper we propose HAMLET, a Hardware-Aware Modular Least Expensive Training framework for real-time domain adaptation. Our approach includes a hardware-aware back-propagation orchestration agent (HAMT) and a dedicated domain-shift detector that enables active control over when and how the model is adapted (LT). Thanks to these advancements, our approach is capable of performing semantic segmentation while simultaneously adapting at more than 29FPS on a single consumer-grade GPU. Our framework's encouraging accuracy and speed trade-off is demonstrated on OnDA and SHIFT benchmarks through experimental results.

Related papers

• TTA-DAME: Test-Time Adaptation with Domain Augmentation and Model Ensemble for Dynamic Driving Conditions [16.21503135588123]
  Test-time Adaptation (TTA) poses a challenge, requiring models to dynamically adapt and perform optimally on shifting target domains.<n>This task is particularly emphasized in real-world driving scenes, where weather domain shifts occur frequently.<n>Our proposed method, TTA-DAME, leverages source domain data augmentation into target domains.
  arXiv  Detail & Related papers  (2025-08-18T07:48:35Z)
• Small Aid, Big Leap: Efficient Test-Time Adaptation for Vision-Language Models with AdaptNet [5.977269026037707]
  Test-time adaptation (TTA) has emerged as a critical technique for enhancing the generalization capability of vision-language models (VLMs) during inference.<n>We introduce SAIL, a novel adapter-based TTA framework that leverages a lightweight, learnable AdaptNet to enable efficient and scalable model adaptation.
  arXiv  Detail & Related papers  (2025-06-03T09:16:51Z)
• What, How, and When Should Object Detectors Update in Continually Changing Test Domains? [34.13756022890991]
  Test-time adaptation algorithms have been proposed to adapt the model online while inferring test data. We propose a novel online adaption approach for object detection in continually changing test domains. Our approach surpasses baselines on widely used benchmarks, achieving improvements of up to 4.9%p and 7.9%p in mAP.
  arXiv  Detail & Related papers  (2023-12-12T07:13:08Z)
• EDAPS: Enhanced Domain-Adaptive Panoptic Segmentation [93.25977558780896]
  We study the panoptic network design and propose a novel architecture (EDAPS) designed explicitly for domain-adaptive panoptic segmentation. EDAPS significantly improves the state-of-the-art performance for panoptic segmentation UDA by a large margin of 20% on SYNTHIA-to-Cityscapes and even 72% on the more challenging SYNTHIA-to-Mapillary Vistas.
  arXiv  Detail & Related papers  (2023-04-27T15:51:19Z)
• Threshold-adaptive Unsupervised Focal Loss for Domain Adaptation of Semantic Segmentation [25.626882426111198]
  Unsupervised domain adaptation (UDA) for semantic segmentation has recently gained increasing research attention. In this paper, we propose a novel two-stage entropy-based UDA method for semantic segmentation. Our method achieves state-of-the-art 58.4% and 59.6% mIoUs on SYNTHIA-to-Cityscapes and GTA5-to-Cityscapes using DeepLabV2 and competitive performance using the lightweight BiSeNet.
  arXiv  Detail & Related papers  (2022-08-23T03:48:48Z)
• UniDAformer: Unified Domain Adaptive Panoptic Segmentation Transformer via Hierarchical Mask Calibration [49.16591283724376]
  We design UniDAformer, a unified domain adaptive panoptic segmentation transformer that is simple but can achieve domain adaptive instance segmentation and semantic segmentation simultaneously within a single network. UniDAformer introduces Hierarchical Mask (HMC) that rectifies inaccurate predictions at the level of regions, superpixels and annotated pixels via online self-training on the fly. It has three unique features: 1) it enables unified domain adaptive panoptic adaptation; 2) it mitigates false predictions and improves domain adaptive panoptic segmentation effectively; 3) it is end-to-end trainable with a much simpler training and inference pipeline.
  arXiv  Detail & Related papers  (2022-06-30T07:32:23Z)
• Towards Online Domain Adaptive Object Detection [79.89082006155135]
  Existing object detection models assume both the training and test data are sampled from the same source domain. We propose a novel unified adaptation framework that adapts and improves generalization on the target domain in online settings.
  arXiv  Detail & Related papers  (2022-04-11T17:47:22Z)
• The Norm Must Go On: Dynamic Unsupervised Domain Adaptation by Normalization [10.274423413222763]
  Domain adaptation is crucial to adapt a learned model to new scenarios, such as domain shifts or changing data distributions. Current approaches usually require a large amount of labeled or unlabeled data from the shifted domain. We propose Dynamic Unsupervised Adaptation (DUA) to overcome this problem.
  arXiv  Detail & Related papers  (2021-12-01T12:43:41Z)
• Unsupervised Domain Adaptation in LiDAR Semantic Segmentation with Self-Supervision and Gated Adapters [13.744866002650076]
  We propose an unsupervised domain adaptation framework that leverages unlabeled target domain data for self-supervision. Experiments adapting from both real-to-real and synthetic-to-real LiDAR semantic segmentation benchmarks demonstrate the significant improvement over prior arts.
  arXiv  Detail & Related papers  (2021-07-20T21:57:18Z)
• Self-supervised Augmentation Consistency for Adapting Semantic Segmentation [56.91850268635183]
  We propose an approach to domain adaptation for semantic segmentation that is both practical and highly accurate. We employ standard data augmentation techniques $-$ photometric noise, flipping and scaling $-$ and ensure consistency of the semantic predictions. We achieve significant improvements of the state-of-the-art segmentation accuracy after adaptation, consistent both across different choices of the backbone architecture and adaptation scenarios.
  arXiv  Detail & Related papers  (2021-04-30T21:32:40Z)
• Unsupervised Domain Adaptation for Spatio-Temporal Action Localization [69.12982544509427]
  S-temporal action localization is an important problem in computer vision. We propose an end-to-end unsupervised domain adaptation algorithm. We show that significant performance gain can be achieved when spatial and temporal features are adapted separately or jointly.
  arXiv  Detail & Related papers  (2020-10-19T04:25:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.