Texture-Shape Bias Balancing for Robust Synthetic-to-Real Semantic Segmentation in Automotive NIR Imagery

• URL: http://arxiv.org/abs/2606.15072v1
• Date: Sat, 13 Jun 2026 03:01:10 GMT
• Title: Texture-Shape Bias Balancing for Robust Synthetic-to-Real Semantic Segmentation in Automotive NIR Imagery
• Abstract summary: We present the first systematic study on synthetic to real domain adaptation for semantic segmentation in NIR images in the automotive domain.<n>We propose a generative augmentation framework that transforms synthetic images into realistic NIR-style variants.<n>Experiments show that balancing inductive bias during training leads to noticeably more robust semantic segmentation.
• Score: 6.537821116235228
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Semantic segmentation is a fundamental component of visual perception in modern automotive systems, enabling pixel-level scene understanding. Near-Infrared imaging (NIR) offers stable detection under difficult illumination conditions, but the development of domain-specific semantic segmentation models remains challenging due to the lack of high-quality annotated data from real-world scenarios. Synthetic datasets offer a scalable alternative, but models trained on synthetic images often suffer performance degradation when transferred to real domains. We present the first systematic study on synthetic to real domain adaptation for semantic segmentation in NIR images in the automotive domain. We propose a generative augmentation framework that transforms synthetic images into realistic NIR-style variants via our introduced target style adaptation (TSA). TSA fine-tunes a latent diffusion model via low-rank adaptation on a small curated set of real NIR images and applies it to synthetic training data using structure-preserving multi-signal conditioning. To reduce texture bias and improve segmentation robustness, we further apply a Voronoi-based style diversification strategy (VSD) that modifies the original textures while preserving scene geometry. Experiments with multiple model architectures on NIR data from vehicle interiors and street scenes show that balancing inductive bias during training leads to noticeably more robust semantic segmentation and effectively reduces the domain gap in our real-world scenarios by up to 63.6% on exterior and 28.4% on interior data. The code is available at GitHub.

Related papers

• The Power of Light: Improving Synthetic-to-Real Domain Adaptation through Physically-Based Indirect Illumination [0.0]
  This paper presents a systematic study analyzing the impact of lighting configurations and background complexity on object detection performance.<n>Through 18 controlled experiments utilizing a state-of-the-art YOLOv12 framework, we demonstrate that complex, indirect lighting configurations paired with domain-relevant background variability significantly increase visual cue richness.<n>Our findings show that avoiding direct specular peaks preserves crucial surface textures, mitigates the domain gap, reduces false positives, and accelerates model convergence compared to using conventional direct-light synthetic data.
  arXiv  Detail & Related papers  (2026-06-21T16:21:34Z)
• Dual-domain Adaptation Networks for Realistic Image Super-resolution [81.34345637776408]
  Realistic image super-resolution (SR) focuses on transforming real-world low-resolution (LR) images into high-resolution (HR) ones.<n>Current methods struggle with limited real-world LR-HR data, impacting the learning of basic image features.<n>We introduce a novel approach, which is able to efficiently adapt pre-trained image SR models from simulated to real-world datasets.
  arXiv  Detail & Related papers  (2025-11-21T12:57:23Z)
• Hybrid CNN-ViT Framework for Motion-Blurred Scene Text Restoration [2.0855516369698845]
  We introduce a hybrid deep learning framework that combines convolutional neural networks (CNNs) with vision transformers (ViTs)<n>The architecture employs a CNN-based encoder-decoder to preserve structural details, while a transformer module enhances global awareness through self-attention.<n>We show that the proposed method attains 32.20 dB in PSNR and 0.934 in SSIM, while remaining lightweight with 2.83 million parameters and an average inference time of 61 ms.
  arXiv  Detail & Related papers  (2025-11-08T17:48:58Z)
• Cityscape-Adverse: Benchmarking Robustness of Semantic Segmentation with Realistic Scene Modifications via Diffusion-Based Image Editing [1.6316196319061707]
  We introduce Cityscape-Adverse, a benchmark that employs diffusion-based image editing to simulate eight adverse conditions. We evaluate the reliability of diffusion-based models in generating realistic scene modifications and assess the performance of CNN and Transformer-based semantic segmentation models. We verify that models trained on Cityscape-Adverse show significantly enhanced resilience when applied to unseen domains.
  arXiv  Detail & Related papers  (2024-11-01T07:47:37Z)
• Is Synthetic Image Useful for Transfer Learning? An Investigation into Data Generation, Volume, and Utilization [62.157627519792946]
  We introduce a novel framework called bridged transfer, which initially employs synthetic images for fine-tuning a pre-trained model to improve its transferability. We propose dataset style inversion strategy to improve the stylistic alignment between synthetic and real images. Our proposed methods are evaluated across 10 different datasets and 5 distinct models, demonstrating consistent improvements.
  arXiv  Detail & Related papers  (2024-03-28T22:25:05Z)
• DNS SLAM: Dense Neural Semantic-Informed SLAM [92.39687553022605]
  DNS SLAM is a novel neural RGB-D semantic SLAM approach featuring a hybrid representation. Our method integrates multi-view geometry constraints with image-based feature extraction to improve appearance details. Our experimental results achieve state-of-the-art performance on both synthetic data and real-world data tracking.
  arXiv  Detail & Related papers  (2023-11-30T21:34:44Z)
• Physics-Driven Turbulence Image Restoration with Stochastic Refinement [80.79900297089176]
  Image distortion by atmospheric turbulence is a critical problem in long-range optical imaging systems. Fast and physics-grounded simulation tools have been introduced to help the deep-learning models adapt to real-world turbulence conditions. This paper proposes the Physics-integrated Restoration Network (PiRN) to help the network to disentangle theity from the degradation and the underlying image.
  arXiv  Detail & Related papers  (2023-07-20T05:49:21Z)
• Alignment-free HDR Deghosting with Semantics Consistent Transformer [76.91669741684173]
  High dynamic range imaging aims to retrieve information from multiple low-dynamic range inputs to generate realistic output. Existing methods often focus on the spatial misalignment across input frames caused by the foreground and/or camera motion. We propose a novel alignment-free network with a Semantics Consistent Transformer (SCTNet) with both spatial and channel attention modules.
  arXiv  Detail & Related papers  (2023-05-29T15:03:23Z)
• Global and Local Texture Randomization for Synthetic-to-Real Semantic Segmentation [40.556020857447535]
  We propose two simple yet effective texture randomization mechanisms, Global Randomization (GTR) and Local Texture Randomization (LTR) GTR is proposed to randomize the texture of source images into diverse texture styles. LTR is proposed to generate diverse local regions for partially stylizing the source images.
  arXiv  Detail & Related papers  (2021-08-05T05:14:49Z)
• Dynamic Object Removal and Spatio-Temporal RGB-D Inpainting via Geometry-Aware Adversarial Learning [9.150245363036165]
  Dynamic objects have a significant impact on the robot's perception of the environment. In this work, we address this problem by synthesizing plausible color, texture and geometry in regions occluded by dynamic objects. We optimize our architecture using adversarial training to synthesize fine realistic textures which enables it to hallucinate color and depth structure in occluded regions online.
  arXiv  Detail & Related papers  (2020-08-12T01:23:21Z)
• Transferring and Regularizing Prediction for Semantic Segmentation [115.88957139226966]
  In this paper, we exploit the intrinsic properties of semantic segmentation to alleviate such problem for model transfer. We present a Regularizer of Prediction Transfer (RPT) that imposes the intrinsic properties as constraints to regularize model transfer in an unsupervised fashion. Extensive experiments are conducted to verify the proposal of RPT on the transfer of models trained on GTA5 and SYNTHIA (synthetic data) to Cityscapes dataset (urban street scenes)
  arXiv  Detail & Related papers  (2020-06-11T16:19:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.