Transferring Styles for Reduced Texture Bias and Improved Robustness in Semantic Segmentation Networks

• URL: http://arxiv.org/abs/2507.10239v1
• Date: Mon, 14 Jul 2025 13:02:19 GMT
• Title: Transferring Styles for Reduced Texture Bias and Improved Robustness in Semantic Segmentation Networks
• Authors: Ben Hamscher, Edgar Heinert, Annika Mütze, Kira Maag, Matthias Rottmann,
• Abstract summary: Recent research has investigated the shape and texture biases of deep neural networks (DNNs) in image classification.<n>We show that training with stylized images reduces texture biases in image classification and improves robustness with respect to image corruptions.<n>In our experiments, it turns out that in semantic segmentation, style transfer augmentation reduces texture bias and strongly increases robustness with respect to common image corruptions as well as adversarial attacks.
• Score: 4.2875024530011085
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent research has investigated the shape and texture biases of deep neural networks (DNNs) in image classification which influence their generalization capabilities and robustness. It has been shown that, in comparison to regular DNN training, training with stylized images reduces texture biases in image classification and improves robustness with respect to image corruptions. In an effort to advance this line of research, we examine whether style transfer can likewise deliver these two effects in semantic segmentation. To this end, we perform style transfer with style varying across artificial image areas. Those random areas are formed by a chosen number of Voronoi cells. The resulting style-transferred data is then used to train semantic segmentation DNNs with the objective of reducing their dependence on texture cues while enhancing their reliance on shape-based features. In our experiments, it turns out that in semantic segmentation, style transfer augmentation reduces texture bias and strongly increases robustness with respect to common image corruptions as well as adversarial attacks. These observations hold for convolutional neural networks and transformer architectures on the Cityscapes dataset as well as on PASCAL Context, showing the generality of the proposed method.

Related papers

• Unwarping Screen Content Images via Structure-texture Enhancement Network and Transformation Self-estimation [2.404130767806698]
  We propose a structure-texture enhancement network (STEN) with transformation self-estimation for screen content images (SCIs)<n>STEN integrates a B-spline implicit neural representation module and a transformation error estimation and self-correction algorithm.<n>Experiments on public SCI datasets demonstrate that our approach significantly outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-04-21T13:59:44Z)
• Classification and regression of trajectories rendered as images via 2D Convolutional Neural Networks [0.0]
  Recent advances in computer vision have facilitated the processing of trajectories rendered as images via artificial neural networks with 2d convolutional layers (CNNs) In this study, we investigate the effectiveness of CNNs for solving classification and regression problems from synthetic trajectories rendered as images using different modalities. Results highlight the importance of choosing an appropriate image resolution according to model depth and motion history in applications where movement direction is critical.
  arXiv  Detail & Related papers  (2024-09-27T15:27:04Z)
• Reducing Texture Bias of Deep Neural Networks via Edge Enhancing Diffusion [4.599829659954009]
  We study the texture bias of CNNs in semantic segmentation. We train CNNs on pre-processed images with less texture to reduce the texture bias. We observe strong texture-dependence of CNNs and moderate texture-dependence of transformers.
  arXiv  Detail & Related papers  (2024-02-14T19:10:40Z)
• Distance Weighted Trans Network for Image Completion [52.318730994423106]
  We propose a new architecture that relies on Distance-based Weighted Transformer (DWT) to better understand the relationships between an image's components. CNNs are used to augment the local texture information of coarse priors. DWT blocks are used to recover certain coarse textures and coherent visual structures.
  arXiv  Detail & Related papers  (2023-10-11T12:46:11Z)
• Segmentation of Roads in Satellite Images using specially modified U-Net CNNs [0.0]
  The aim of this paper is to build an image classifier for satellite images of urban scenes that identifies the portions of the images in which a road is located. Unlike conventional computer vision algorithms, convolutional neural networks (CNNs) provide accurate and reliable results on this task.
  arXiv  Detail & Related papers  (2021-09-29T19:08:32Z)
• Spatially-Adaptive Image Restoration using Distortion-Guided Networks [51.89245800461537]
  We present a learning-based solution for restoring images suffering from spatially-varying degradations. We propose SPAIR, a network design that harnesses distortion-localization information and dynamically adjusts to difficult regions in the image.
  arXiv  Detail & Related papers  (2021-08-19T11:02:25Z)
• Controllable Person Image Synthesis with Spatially-Adaptive Warped Normalization [72.65828901909708]
  Controllable person image generation aims to produce realistic human images with desirable attributes. We introduce a novel Spatially-Adaptive Warped Normalization (SAWN), which integrates a learned flow-field to warp modulation parameters. We propose a novel self-training part replacement strategy to refine the pretrained model for the texture-transfer task.
  arXiv  Detail & Related papers  (2021-05-31T07:07:44Z)
• Does enhanced shape bias improve neural network robustness to common corruptions? [14.607217936005817]
  Recent work indicates that CNNs trained on ImageNet are biased towards features that encode textures. It has been shown that augmenting the training data with different image styles decreases this texture bias in favor of increased shape bias. We perform a systematic study of different ways of composing inputs based on natural images, explicit edge information, and stylization.
  arXiv  Detail & Related papers  (2021-04-20T07:06:53Z)
• Encoding Robustness to Image Style via Adversarial Feature Perturbations [72.81911076841408]
  We adapt adversarial training by directly perturbing feature statistics, rather than image pixels, to produce robust models. Our proposed method, Adversarial Batch Normalization (AdvBN), is a single network layer that generates worst-case feature perturbations during training.
  arXiv  Detail & Related papers  (2020-09-18T17:52:34Z)
• Transferring and Regularizing Prediction for Semantic Segmentation [115.88957139226966]
  In this paper, we exploit the intrinsic properties of semantic segmentation to alleviate such problem for model transfer. We present a Regularizer of Prediction Transfer (RPT) that imposes the intrinsic properties as constraints to regularize model transfer in an unsupervised fashion. Extensive experiments are conducted to verify the proposal of RPT on the transfer of models trained on GTA5 and SYNTHIA (synthetic data) to Cityscapes dataset (urban street scenes)
  arXiv  Detail & Related papers  (2020-06-11T16:19:41Z)
• Image Fine-grained Inpainting [89.17316318927621]
  We present a one-stage model that utilizes dense combinations of dilated convolutions to obtain larger and more effective receptive fields. To better train this efficient generator, except for frequently-used VGG feature matching loss, we design a novel self-guided regression loss. We also employ a discriminator with local and global branches to ensure local-global contents consistency.
  arXiv  Detail & Related papers  (2020-02-07T03:45:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.