Reimagine BiSeNet for Real-Time Domain Adaptation in Semantic Segmentation

• URL: http://arxiv.org/abs/2110.11662v1
• Date: Fri, 22 Oct 2021 08:39:28 GMT
• Title: Reimagine BiSeNet for Real-Time Domain Adaptation in Semantic Segmentation
• Authors: Antonio Tavera, Carlo Masone, Barbara Caputo
• Abstract summary: We look at the challenge of real-time semantic segmentation across domains. We train a model to act appropriately on real-world data even though it was trained on a synthetic realm. We employ a new lightweight and shallow discriminator that was specifically created for this purpose.
• Score: 17.761939190746812
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Semantic segmentation models have reached remarkable performance across various tasks. However, this performance is achieved with extremely large models, using powerful computational resources and without considering training and inference time. Real-world applications, on the other hand, necessitate models with minimal memory demands, efficient inference speed, and executable with low-resources embedded devices, such as self-driving vehicles. In this paper, we look at the challenge of real-time semantic segmentation across domains, and we train a model to act appropriately on real-world data even though it was trained on a synthetic realm. We employ a new lightweight and shallow discriminator that was specifically created for this purpose. To the best of our knowledge, we are the first to present a real-time adversarial approach for assessing the domain adaption problem in semantic segmentation. We tested our framework in the two standard protocol: GTA5 to Cityscapes and SYNTHIA to Cityscapes. Code is available at: https://github.com/taveraantonio/RTDA.

Related papers

• Alice Benchmarks: Connecting Real World Re-Identification with the Synthetic [92.02220105679713]
  We introduce the Alice benchmarks, large-scale datasets providing benchmarks and evaluation protocols to the research community. Within the Alice benchmarks, two object re-ID tasks are offered: person and vehicle re-ID. As an important feature of our real target, the clusterability of its training set is not manually guaranteed to make it closer to a real domain adaptation test scenario.
  arXiv  Detail & Related papers  (2023-10-06T17:58:26Z)
• You Only Look at Once for Real-time and Generic Multi-Task [20.61477620156465]
  A-YOLOM is an adaptive, real-time, and lightweight multi-task model. We develop an end-to-end multi-task model with a unified and streamlined segmentation structure. We achieve competitive results on the BDD100k dataset.
  arXiv  Detail & Related papers  (2023-10-02T21:09:43Z)
• One-Shot Domain Adaptive and Generalizable Semantic Segmentation with Class-Aware Cross-Domain Transformers [96.51828911883456]
  Unsupervised sim-to-real domain adaptation (UDA) for semantic segmentation aims to improve the real-world test performance of a model trained on simulated data. Traditional UDA often assumes that there are abundant unlabeled real-world data samples available during training for the adaptation. We explore the one-shot unsupervised sim-to-real domain adaptation (OSUDA) and generalization problem, where only one real-world data sample is available.
  arXiv  Detail & Related papers  (2022-12-14T15:54:15Z)
• RTFormer: Efficient Design for Real-Time Semantic Segmentation with Transformer [63.25665813125223]
  We propose RTFormer, an efficient dual-resolution transformer for real-time semantic segmenation. It achieves better trade-off between performance and efficiency than CNN-based models. Experiments on mainstream benchmarks demonstrate the effectiveness of our proposed RTFormer.
  arXiv  Detail & Related papers  (2022-10-13T16:03:53Z)
• On Efficient Real-Time Semantic Segmentation: A Survey [12.404169549562523]
  We take a look at the works that aim to address this misalignment with more compact and efficient models capable of deployment on low-memory embedded systems. We evaluate the inference speed of the discussed models under consistent hardware and software setups. Our experimental results demonstrate that many works are capable of real-time performance on resource-constrained hardware, while illustrating the consistent trade-off between latency and accuracy.
  arXiv  Detail & Related papers  (2022-06-17T08:00:27Z)
• Unseen Object Instance Segmentation with Fully Test-time RGB-D Embeddings Adaptation [14.258456366985444]
  Recently, a popular solution is leveraging RGB-D features of large-scale synthetic data and applying the model to unseen real-world scenarios. We re-emphasize the adaptation process across Sim2Real domains in this paper. We propose a framework to conduct the Fully Test-time RGB-D Embeddings Adaptation (FTEA) based on parameters of the BatchNorm layer.
  arXiv  Detail & Related papers  (2022-04-21T02:35:20Z)
• Towards Optimal Strategies for Training Self-Driving Perception Models in Simulation [98.51313127382937]
  We focus on the use of labels in the synthetic domain alone. Our approach introduces both a way to learn neural-invariant representations and a theoretically inspired view on how to sample the data from the simulator. We showcase our approach on the bird's-eye-view vehicle segmentation task with multi-sensor data.
  arXiv  Detail & Related papers  (2021-11-15T18:37:43Z)
• PixMatch: Unsupervised Domain Adaptation via Pixelwise Consistency Training [4.336877104987131]
  Unsupervised domain adaptation is a promising technique for semantic segmentation. We present a novel framework for unsupervised domain adaptation based on the notion of target-domain consistency training. Our approach is simpler, easier to implement, and more memory-efficient during training.
  arXiv  Detail & Related papers  (2021-05-17T19:36:28Z)
• LevelSet R-CNN: A Deep Variational Method for Instance Segmentation [79.20048372891935]
  Currently, many state of the art models are based on the Mask R-CNN framework. We propose LevelSet R-CNN, which combines the best of both worlds by obtaining powerful feature representations. We demonstrate the effectiveness of our approach on COCO and Cityscapes datasets.
  arXiv  Detail & Related papers  (2020-07-30T17:52:18Z)
• Learning Fast and Robust Target Models for Video Object Segmentation [83.3382606349118]
  Video object segmentation (VOS) is a highly challenging problem since the initial mask, defining the target object, is only given at test-time. Most previous approaches fine-tune segmentation networks on the first frame, resulting in impractical frame-rates and risk of overfitting. We propose a novel VOS architecture consisting of two network components.
  arXiv  Detail & Related papers  (2020-02-27T21:58:06Z)
• Unsupervised Domain Adaptation for Mobile Semantic Segmentation based on Cycle Consistency and Feature Alignment [28.61782696432711]
  We propose a novel Unsupervised Domain Adaptation (UDA) strategy to address the domain shift issue between real world and synthetic representations. We show how the proposed strategy is able to obtain impressive performance in adapting a segmentation network trained on synthetic data to real world scenarios.
  arXiv  Detail & Related papers  (2020-01-14T10:12:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.